[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:41:58 dist-test-slave-gkw7 krb5kdc[32006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:41:58 dist-test-slave-gkw7 krb5kdc[32006](info): set up 2 sockets
Aug 09 08:41:58 dist-test-slave-gkw7 krb5kdc[32006](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 08:42:00.035260 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:42:00.165371 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:41661
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:41661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:00.251562 32025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:00.251780 32025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:00.251802 32025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:00.252909 32025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:00.252946 32025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:00.252961 32025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:00.252974 32025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:00.252985 32025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:00.254601 32025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:41661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:41661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:00.254931 32025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:00.255160 32025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:42:00.261539 32025 server_base.cc:1034] running on GCE node
W20260809 08:42:00.262982 32032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:00.263089 32031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:00.263233 32034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:00.263569 32025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:00.264050 32025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:00.265229 32025 hybrid_clock.cc:648] HybridClock initialized: now 1786264920265222 us; error 100 us; skew 500 ppm
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:00.271337 32025 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:42:00.271967 32025 webserver.cc:460] Webserver started at http://127.31.63.126:34721/ using document root <none> and password file <none>
I20260809 08:42:00.272249 32025 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:00.272297 32025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:00.272425 32025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:00.273717 32025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "6656f79bdcb44de889a35e50a0bc796e"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "b1465762cbf45dcdb6acc54fe2566ac4"
server_key_iv: "3234b2f1f2e739583d003e90eadfd173"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.274102 32025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "6656f79bdcb44de889a35e50a0bc796e"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "b1465762cbf45dcdb6acc54fe2566ac4"
server_key_iv: "3234b2f1f2e739583d003e90eadfd173"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.276046 32025 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 08:42:00.277081 32045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:00.277401 32025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:00.277496 32025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "6656f79bdcb44de889a35e50a0bc796e"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "b1465762cbf45dcdb6acc54fe2566ac4"
server_key_iv: "3234b2f1f2e739583d003e90eadfd173"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.277603 32025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:00.317991 32025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:00.320370 32025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:00.320557 32025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:00.325615 32025 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:41661
I20260809 08:42:00.325706 32122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:41661 every 8 connection(s)
I20260809 08:42:00.326365 32025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:00.327760 32123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:00.330976 32123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e: Bootstrap starting.
I20260809 08:42:00.331342 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 32025
I20260809 08:42:00.331480 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:00.332139 32123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:00.332456 31997 external_mini_cluster.cc:1428] Setting key 9b6c7d48e1de77e79c86ef65c87c40ee
I20260809 08:42:00.332602 32123 log.cc:824] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:00.333678 32123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e: No bootstrap required, opened a new log
I20260809 08:42:00.336398 32123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6656f79bdcb44de889a35e50a0bc796e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 41661 } }
I20260809 08:42:00.336627 32123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:00.336773 32123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6656f79bdcb44de889a35e50a0bc796e, State: Initialized, Role: FOLLOWER
I20260809 08:42:00.336992 32123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6656f79bdcb44de889a35e50a0bc796e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 41661 } }
I20260809 08:42:00.337158 32123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:00.337186 32123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:00.337299 32123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:00.338472 32123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6656f79bdcb44de889a35e50a0bc796e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 41661 } }
I20260809 08:42:00.338704 32123 leader_election.cc:302] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6656f79bdcb44de889a35e50a0bc796e; no voters:
I20260809 08:42:00.338967 32123 leader_election.cc:288] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:00.338987 32130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:00.339402 32123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:00.341867 32130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [term 1 LEADER]: Becoming Leader. State: Replica: 6656f79bdcb44de889a35e50a0bc796e, State: Running, Role: LEADER
I20260809 08:42:00.342010 32130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6656f79bdcb44de889a35e50a0bc796e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 41661 } }
I20260809 08:42:00.342607 32147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6656f79bdcb44de889a35e50a0bc796e. Latest consensus state: current_term: 1 leader_uuid: "6656f79bdcb44de889a35e50a0bc796e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6656f79bdcb44de889a35e50a0bc796e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 41661 } } }
I20260809 08:42:00.342715 32147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:00.342734 32131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6656f79bdcb44de889a35e50a0bc796e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6656f79bdcb44de889a35e50a0bc796e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 41661 } } }
I20260809 08:42:00.342823 32131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e [sys.catalog]: This master's current role is: LEADER
W20260809 08:42:00.344368 32152 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:42:00.344460 32152 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:42:00.344617 32150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:00.344976 32150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:00.347517 32150 catalog_manager.cc:1347] Generated new cluster ID: 590c5e239e764415a1015adf6f3fe78e
I20260809 08:42:00.347571 32150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:00.350891 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:00.333925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36440 (local address 127.31.63.126:41661)
0809 08:42:00.336400 (+ 2475us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:00.336408 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:00.336436 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:42:00.337041 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:00.337046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:00.337124 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:00.337280 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:00.339183 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.340437 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.341465 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.342009 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.342124 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:00.346605 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:00.346609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:00.346609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:00.346621 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:00.348784 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.349244 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.349247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.349247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.349360 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.349642 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.349643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.349643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.349926 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:00.350059 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:00.350449 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 08:42:00.350643 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2371,"thread_start_us":128,"threads_started":1}
I20260809 08:42:00.368891 32150 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:00.369622 32150 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:00.376827 32150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6656f79bdcb44de889a35e50a0bc796e: Generated new TSK 0
I20260809 08:42:00.377048 32150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:42:00.412977 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:41661
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:00.502681 32159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:00.502934 32159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:00.502975 32159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:00.504124 32159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:00.504199 32159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:00.504257 32159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:00.505813 32159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:41661
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:00.506140 32159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:00.506392 32159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:00.512715 32167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:00.512734 32166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:00.512766 32170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:00.513051 32159 server_base.cc:1034] running on GCE node
I20260809 08:42:00.513293 32159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:00.513592 32159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:00.514755 32159 hybrid_clock.cc:648] HybridClock initialized: now 1786264920514735 us; error 37 us; skew 500 ppm
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:00.517442 32159 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:42:00.518024 32159 webserver.cc:460] Webserver started at http://127.31.63.65:35209/ using document root <none> and password file <none>
I20260809 08:42:00.518311 32159 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:00.518358 32159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:00.518512 32159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:00.519748 32159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "47662a91fadb470ab8e29068245d9ac8"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "1a9d67a10b06091a8947fa4e7940a3bd"
server_key_iv: "e01ee47401f7f76d298a25e48080f86e"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.520140 32159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "47662a91fadb470ab8e29068245d9ac8"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "1a9d67a10b06091a8947fa4e7940a3bd"
server_key_iv: "e01ee47401f7f76d298a25e48080f86e"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.521620 32159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:42:00.522441 32178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:00.522610 32159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:00.522701 32159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "47662a91fadb470ab8e29068245d9ac8"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "1a9d67a10b06091a8947fa4e7940a3bd"
server_key_iv: "e01ee47401f7f76d298a25e48080f86e"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.522771 32159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:00.532426 32159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:00.534740 32159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:00.534906 32159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:00.535192 32159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:00.535588 32159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:00.535656 32159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:00.535681 32159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:00.535693 32159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:00.544902 32159 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:43987
I20260809 08:42:00.544981 32344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:43987 every 8 connection(s)
I20260809 08:42:00.545379 32159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:00.549414 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 32159
I20260809 08:42:00.549538 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:00.549746 31997 external_mini_cluster.cc:1428] Setting key 30b74d8b212c2330a36dd064536a8997
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:00.556445 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:00.546523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:39439 (local address 127.31.63.126:41661)
0809 08:42:00.546587 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:00.546591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:00.546831 (+ 240us) server_negotiation.cc:407] Connection header received
0809 08:42:00.547485 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:00.547487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:00.547506 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:00.547562 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:00.548687 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.549222 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.550658 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.550870 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.550957 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:00.553438 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:00.553439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:00.553439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:00.553448 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:00.554977 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.555415 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.555416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.555417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.555478 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.555748 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.555749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.555749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.555875 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:00.555944 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:00.556242 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 08:42:00.556350 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:42:00.556655 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:00.546510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:41661 (local address 127.31.63.65:39439)
0809 08:42:00.546694 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:00.546738 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:00.547371 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:00.547636 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:00.547641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:00.548358 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:00.548568 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:00.548574 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.549331 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.549332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:00.550586 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:00.550589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.550973 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.550974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:00.551114 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:00.551682 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:00.551692 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:00.553315 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:00.555076 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:00.555092 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:00.555094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:00.555332 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:00.555554 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:00.555554 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:00.555555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:00.555669 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:00.555948 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:00.555956 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:00.556108 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 08:42:00.556220 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 08:42:00.556403 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":85,"threads_started":1}
I20260809 08:42:00.557277 32345 heartbeater.cc:347] Connected to a master server at 127.31.63.126:41661
I20260809 08:42:00.557377 32345 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:00.557636 32345 heartbeater.cc:511] Master 127.31.63.126:41661 requested a full tablet report, sending...
I20260809 08:42:00.558676 32070 ts_manager.cc:194] Registered new tserver with Master: 47662a91fadb470ab8e29068245d9ac8 (127.31.63.65:43987)
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
I20260809 08:42:00.559873 32070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:39439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:42:00.580603 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:41661
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:00.667569 32351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:00.667783 32351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:00.667807 32351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:00.668886 32351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:00.668920 32351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:00.668947 32351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:00.670463 32351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:41661
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:00.670666 32351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:00.670848 32351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:00.676333 32361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:00.676354 32357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:00.676389 32363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:00.676975 32351 server_base.cc:1034] running on GCE node
I20260809 08:42:00.677253 32351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:00.677506 32351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:00.678704 32351 hybrid_clock.cc:648] HybridClock initialized: now 1786264920678683 us; error 37 us; skew 500 ppm
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:00.681421 32351 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:42:00.682018 32351 webserver.cc:460] Webserver started at http://127.31.63.66:38095/ using document root <none> and password file <none>
I20260809 08:42:00.682263 32351 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:00.682336 32351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:00.682493 32351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:00.683638 32351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "824ca131b74d4f29b9b114127b6ad14d"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "72ebab32b6a9964ffd8f75f4fb96038f"
server_key_iv: "ad8dd17933f41ab581114eed47e2f08f"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.684053 32351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "824ca131b74d4f29b9b114127b6ad14d"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "72ebab32b6a9964ffd8f75f4fb96038f"
server_key_iv: "ad8dd17933f41ab581114eed47e2f08f"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.685495 32351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260809 08:42:00.686395 32373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:00.686594 32351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:00.686699 32351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "824ca131b74d4f29b9b114127b6ad14d"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "72ebab32b6a9964ffd8f75f4fb96038f"
server_key_iv: "ad8dd17933f41ab581114eed47e2f08f"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.686791 32351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:00.711954 32351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:00.714314 32351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:00.714489 32351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:00.714768 32351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:00.715235 32351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:00.715299 32351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:00.715348 32351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:00.715372 32351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:00.723989 32351 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:33375
I20260809 08:42:00.724043 32548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:33375 every 8 connection(s)
I20260809 08:42:00.724475 32351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:42:00.726874 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 32351
I20260809 08:42:00.726948 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:42:00.727149 31997 external_mini_cluster.cc:1428] Setting key 58c181189c83bc65d7a55fded1bc29a5
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:00.735380 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:00.725295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:51795 (local address 127.31.63.126:41661)
0809 08:42:00.725348 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:00.725349 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:00.725622 (+ 273us) server_negotiation.cc:407] Connection header received
0809 08:42:00.726147 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:00.726148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:00.726159 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:00.726227 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:00.727448 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.728063 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.729088 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.729317 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.729397 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:00.732138 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:00.732140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:00.732140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:00.732150 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:00.733802 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.734306 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.734308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.734308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.734377 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.734665 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.734666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.734666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.734797 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:00.734882 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:00.735187 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 08:42:00.735280 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:42:00.735622 32552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:00.725266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:41661 (local address 127.31.63.66:51795)
0809 08:42:00.725437 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:00.725481 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:00.726057 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:00.726286 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:00.726290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:00.726992 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:00.727249 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:00.727255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.728168 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.728169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:00.728991 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:00.728993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.729406 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.729407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:00.729579 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:00.730242 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:00.730254 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:00.731924 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:00.733930 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:00.733939 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:00.733942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:00.734204 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:00.734454 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:00.734455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:00.734455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:00.734586 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:00.734873 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:00.734881 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:00.735034 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 08:42:00.735163 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 08:42:00.735341 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260809 08:42:00.736099 32550 heartbeater.cc:347] Connected to a master server at 127.31.63.126:41661
I20260809 08:42:00.736290 32550 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:00.736511 32550 heartbeater.cc:511] Master 127.31.63.126:41661 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
I20260809 08:42:00.737022 32070 ts_manager.cc:194] Registered new tserver with Master: 824ca131b74d4f29b9b114127b6ad14d (127.31.63.66:33375)
I20260809 08:42:00.737465 32070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:51795
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:42:00.756879 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:41661
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:00.845232 32555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:00.845446 32555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:00.845472 32555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:00.846668 32555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:00.846721 32555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:00.846750 32555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:00.848249 32555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:41661
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:00.848510 32555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:00.848767 32555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:00.854228 32566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:00.854212 32563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:00.854550 32555 server_base.cc:1034] running on GCE node
W20260809 08:42:00.854210 32562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:00.854903 32555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:00.855119 32555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:00.856289 32555 hybrid_clock.cc:648] HybridClock initialized: now 1786264920856309 us; error 60 us; skew 500 ppm
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:00.858954 32555 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:42:00.859473 32555 webserver.cc:460] Webserver started at http://127.31.63.67:37293/ using document root <none> and password file <none>
I20260809 08:42:00.859738 32555 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:00.859812 32555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:00.859963 32555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:00.861109 32555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "c3a1ee3fd6984c0c822d6b188f5ada58"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "4cf8bcb136d60ea34d36c97baa76634d"
server_key_iv: "853d4ea808df80b5fe99c1ce4f8b0865"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.861480 32555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "c3a1ee3fd6984c0c822d6b188f5ada58"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "4cf8bcb136d60ea34d36c97baa76634d"
server_key_iv: "853d4ea808df80b5fe99c1ce4f8b0865"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.862973 32555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:00.863720 32579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:00.863899 32555 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:00.864008 32555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "c3a1ee3fd6984c0c822d6b188f5ada58"
format_stamp: "Formatted at 2026-08-09 08:42:00 on dist-test-slave-gkw7"
server_key: "4cf8bcb136d60ea34d36c97baa76634d"
server_key_iv: "853d4ea808df80b5fe99c1ce4f8b0865"
server_key_version: "encryptionkey@0"
I20260809 08:42:00.864084 32555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:00.875051 32555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:00.877518 32555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:00.877758 32555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:00.878093 32555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:00.878474 32555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:00.878531 32555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:00.878575 32555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:00.878623 32555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:00.887894 32555 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:45411
I20260809 08:42:00.887966 32757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:45411 every 8 connection(s)
I20260809 08:42:00.888307 32555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:42:00.892732 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 32555
I20260809 08:42:00.892829 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:00.893019 31997 external_mini_cluster.cc:1428] Setting key 66d2969b1cfc2489671ce351805c4967
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:00.898691 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:00.889117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:45033 (local address 127.31.63.126:41661)
0809 08:42:00.889187 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:00.889191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:00.889464 (+ 273us) server_negotiation.cc:407] Connection header received
0809 08:42:00.890064 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:00.890065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:00.890076 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:00.890133 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:00.891184 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.891700 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.892830 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.893074 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.893143 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:00.895752 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:00.895753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:00.895753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:00.895760 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:00.897299 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.897754 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.897755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.897755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.897806 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.898058 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.898058 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.898058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.898165 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:00.898212 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:00.898531 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 08:42:00.898623 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:42:00.898809 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:00.889145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:41661 (local address 127.31.63.67:45033)
0809 08:42:00.889323 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:00.889369 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:00.889972 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:00.890197 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:00.890201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:00.890875 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:00.891086 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:00.891092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.891791 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.891792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:00.892715 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:00.892719 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.893175 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.893177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:00.893307 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:00.893967 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:00.893977 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:00.895649 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:00.897384 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:00.897392 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:00.897394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:00.897690 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:00.897886 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:00.897887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:00.897887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:00.898006 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:00.898218 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:00.898224 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:00.898361 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 08:42:00.898445 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:42:00.898587 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260809 08:42:00.899359 32758 heartbeater.cc:347] Connected to a master server at 127.31.63.126:41661
I20260809 08:42:00.899471 32758 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:00.899648 32758 heartbeater.cc:511] Master 127.31.63.126:41661 requested a full tablet report, sending...
I20260809 08:42:00.900015 32070 ts_manager.cc:194] Registered new tserver with Master: c3a1ee3fd6984c0c822d6b188f5ada58 (127.31.63.67:45411)
I20260809 08:42:00.900372 32070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:45033
I20260809 08:42:00.905400 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:42:00.930984 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:00.923244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.31.63.65:43987)
0809 08:42:00.923438 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:00.923443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:00.923455 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:42:00.923483 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:00.923485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:00.923500 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:00.923581 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:00.924123 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.924930 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.925761 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.926138 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.926194 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:00.928446 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:00.928454 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:00.928454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:00.928462 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:00.929961 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.930281 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.930284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.930284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.930339 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.930498 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.930499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.930499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.930623 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:00.930749 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:00.930784 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 08:42:00.930847 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":107,"threads_started":1}
W20260809 08:42:00.931478 32223 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59490
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:42:00 dist-test-slave-gkw7 krb5kdc[32006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:42:00.955514 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:00.948688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59506 (local address 127.31.63.65:43987)
0809 08:42:00.948758 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:00.948760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:00.948806 (+ 46us) server_negotiation.cc:407] Connection header received
0809 08:42:00.948883 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:00.948884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:00.948899 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:00.948954 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:00.949669 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.950172 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.950872 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:00.951068 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:00.951118 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:00.953336 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:00.953337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:00.953337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:00.953344 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:00.954592 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.954888 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.954889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.954889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.954940 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:00.955100 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:00.955100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:00.955101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:00.955198 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:00.955284 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:00.955356 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 08:42:00.955418 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:42:00.956135 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 32159
I20260809 08:42:00.967046 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 32351
I20260809 08:42:00.990507 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 32555
I20260809 08:42:01.001242 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 32025
2026-08-09T08:42:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2229 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:42:01 dist-test-slave-gkw7 krb5kdc[310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:42:01 dist-test-slave-gkw7 krb5kdc[310](info): set up 2 sockets
Aug 09 08:42:01 dist-test-slave-gkw7 krb5kdc[310](info): commencing operation
krb5kdc: starting...
W20260809 08:42:02.231066 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:42:02 dist-test-slave-gkw7 krb5kdc[310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:42:02.354310 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:34927
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:34927
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:02.438406 329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:02.438670 329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:02.438716 329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:02.439845 329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:02.439884 329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:02.439898 329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:02.439947 329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:02.439970 329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:02.441438 329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:34927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:34927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:02.441725 329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:02.441977 329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:02.446789 339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:02.446801 336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:02.446867 337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:02.446944 329 server_base.cc:1034] running on GCE node
I20260809 08:42:02.447305 329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:02.447633 329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:02.448803 329 hybrid_clock.cc:648] HybridClock initialized: now 1786264922448789 us; error 27 us; skew 500 ppm
Aug 09 08:42:02 dist-test-slave-gkw7 krb5kdc[310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264922, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:02.451254 329 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:42:02.451728 329 webserver.cc:460] Webserver started at http://127.31.63.126:40629/ using document root <none> and password file <none>
I20260809 08:42:02.451969 329 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:02.452039 329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:02.452193 329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:02.453238 329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "50429ee81d534b0b95bec6dc9e058b21"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "c5f3aca9cf93b465642c069f88e15843"
server_key_iv: "4318601b9542646cb470908e7f1b0470"
server_key_version: "encryptionkey@0"
I20260809 08:42:02.453668 329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "50429ee81d534b0b95bec6dc9e058b21"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "c5f3aca9cf93b465642c069f88e15843"
server_key_iv: "4318601b9542646cb470908e7f1b0470"
server_key_version: "encryptionkey@0"
I20260809 08:42:02.455024 329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:02.455737 350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:02.455916 329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:02.456017 329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "50429ee81d534b0b95bec6dc9e058b21"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "c5f3aca9cf93b465642c069f88e15843"
server_key_iv: "4318601b9542646cb470908e7f1b0470"
server_key_version: "encryptionkey@0"
I20260809 08:42:02.456101 329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:02.480159 329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:02.482244 329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:02.482414 329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:02.486761 329 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:34927
I20260809 08:42:02.486799 430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:34927 every 8 connection(s)
I20260809 08:42:02.487190 329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:02.488080 431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:02.489391 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 329
I20260809 08:42:02.489490 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:02.489687 31997 external_mini_cluster.cc:1428] Setting key efd98683e5b99e4f4e062cb5a2cb7269
I20260809 08:42:02.490885 431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21: Bootstrap starting.
I20260809 08:42:02.491672 431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:02.492025 431 log.cc:824] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:02.492782 431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21: No bootstrap required, opened a new log
I20260809 08:42:02.495353 431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50429ee81d534b0b95bec6dc9e058b21" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34927 } }
I20260809 08:42:02.495537 431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:02.495576 431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50429ee81d534b0b95bec6dc9e058b21, State: Initialized, Role: FOLLOWER
I20260809 08:42:02.495721 431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50429ee81d534b0b95bec6dc9e058b21" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34927 } }
I20260809 08:42:02.495824 431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:02.495860 431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:02.495911 431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:02.496649 431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50429ee81d534b0b95bec6dc9e058b21" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34927 } }
I20260809 08:42:02.496747 431 leader_election.cc:302] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50429ee81d534b0b95bec6dc9e058b21; no voters:
I20260809 08:42:02.496942 431 leader_election.cc:288] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:02.496994 440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [term 1 FOLLOWER]: Leader election won for term 1
Aug 09 08:42:02 dist-test-slave-gkw7 krb5kdc[310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:02.497149 440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [term 1 LEADER]: Becoming Leader. State: Replica: 50429ee81d534b0b95bec6dc9e058b21, State: Running, Role: LEADER
I20260809 08:42:02.497239 431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:02.497365 440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50429ee81d534b0b95bec6dc9e058b21" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34927 } }
I20260809 08:42:02.497789 440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50429ee81d534b0b95bec6dc9e058b21. Latest consensus state: current_term: 1 leader_uuid: "50429ee81d534b0b95bec6dc9e058b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50429ee81d534b0b95bec6dc9e058b21" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34927 } } }
I20260809 08:42:02.497799 444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50429ee81d534b0b95bec6dc9e058b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50429ee81d534b0b95bec6dc9e058b21" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34927 } } }
I20260809 08:42:02.497991 440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:02.498004 444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:02.498250 451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:02.498857 451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:02.500628 451 catalog_manager.cc:1347] Generated new cluster ID: 622f620cf7ea4774b65ce3f83622f973
I20260809 08:42:02.500682 451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:02.501665 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:02.490273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.31.63.126:34927)
0809 08:42:02.490468 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:02.490477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:02.490503 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:42:02.490915 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:02.490921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:02.491011 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:02.491142 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:02.491929 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:02.493088 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:02.494177 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:02.494573 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:02.494670 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:02.497635 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:02.497639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:02.497639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:02.497651 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:02.499910 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:02.500334 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:02.500337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:02.500338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:02.500456 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:02.500763 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:02.500764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:02.500765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:02.501004 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:02.501151 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:02.501301 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 08:42:02.501443 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260809 08:42:02.517369 451 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:02.517953 451 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:02.522378 451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50429ee81d534b0b95bec6dc9e058b21: Generated new TSK 0
I20260809 08:42:02.522564 451 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:42:02.556828 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34927
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:02.651614 471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:02.651842 471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:02.651868 471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:02.653183 471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:02.653234 471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:02.653261 471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:02.654803 471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34927
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:02.655025 471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:02.655218 471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:02.661971 480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:02.661973 479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:02.662143 483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:02.662935 471 server_base.cc:1034] running on GCE node
I20260809 08:42:02.663203 471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:02.663503 471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:02.664674 471 hybrid_clock.cc:648] HybridClock initialized: now 1786264922664647 us; error 42 us; skew 500 ppm
Aug 09 08:42:02 dist-test-slave-gkw7 krb5kdc[310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264922, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:02.668120 471 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:42:02.668690 471 webserver.cc:460] Webserver started at http://127.31.63.65:32945/ using document root <none> and password file <none>
I20260809 08:42:02.668960 471 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:02.669031 471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:02.669174 471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:02.670351 471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "a1172095c2964457989e800df407db34"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "0b0bdb277e23f74aeefef5f4171fc4e1"
server_key_iv: "d82b0e9f5aeb58075feb2b34efb8ca77"
server_key_version: "encryptionkey@0"
I20260809 08:42:02.670744 471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "a1172095c2964457989e800df407db34"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "0b0bdb277e23f74aeefef5f4171fc4e1"
server_key_iv: "d82b0e9f5aeb58075feb2b34efb8ca77"
server_key_version: "encryptionkey@0"
I20260809 08:42:02.672143 471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:42:02.672941 495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:02.673130 471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:02.673242 471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "a1172095c2964457989e800df407db34"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "0b0bdb277e23f74aeefef5f4171fc4e1"
server_key_iv: "d82b0e9f5aeb58075feb2b34efb8ca77"
server_key_version: "encryptionkey@0"
I20260809 08:42:02.673319 471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:02.692008 471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:02.694554 471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:02.694788 471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:02.695044 471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:02.695366 471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:02.695423 471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:02.695484 471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:02.695515 471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:02.702425 471 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:33051
I20260809 08:42:02.702479 656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:33051 every 8 connection(s)
I20260809 08:42:02.702816 471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:02.703305 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 471
I20260809 08:42:02.703418 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:02.703608 31997 external_mini_cluster.cc:1428] Setting key 2121f10d5409dd60c4d4dfde3d35eecb
Aug 09 08:42:02 dist-test-slave-gkw7 krb5kdc[310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264922, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
I20260809 08:42:02.714479 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:02.703573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:34929 (local address 127.31.63.126:34927)
0809 08:42:02.703627 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:02.703630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:02.703953 (+ 323us) server_negotiation.cc:407] Connection header received
0809 08:42:02.705582 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:02.705583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:02.705599 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:02.705701 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:02.706643 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:02.707160 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:02.708334 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:02.708569 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:02.708657 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:02.711269 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:02.711271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:02.711271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:02.711280 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:02.712957 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:02.713415 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:02.713417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:02.713417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:02.713481 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:02.713785 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:02.713786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:02.713786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:02.713913 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:02.713990 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:02.714277 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 08:42:02.714372 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:42:02.714707 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:02.703575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:34927 (local address 127.31.63.65:34929)
0809 08:42:02.703785 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:02.703837 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:02.705417 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:02.705795 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:02.705800 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:02.706226 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:02.706537 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:02.706543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:02.707264 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:02.707266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:02.708235 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:02.708238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:02.708672 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:02.708673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:02.708861 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:02.709451 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:02.709460 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:02.711167 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:02.713045 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:02.713053 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:02.713055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:02.713329 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:02.713575 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:02.713576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:02.713576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:02.713696 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:02.713996 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:02.714004 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:02.714136 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 08:42:02.714253 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 08:42:02.714458 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":95,"threads_started":1}
I20260809 08:42:02.715281 658 heartbeater.cc:347] Connected to a master server at 127.31.63.126:34927
I20260809 08:42:02.715390 658 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:02.715601 658 heartbeater.cc:511] Master 127.31.63.126:34927 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
I20260809 08:42:02.716354 374 ts_manager.cc:194] Registered new tserver with Master: a1172095c2964457989e800df407db34 (127.31.63.65:33051)
I20260809 08:42:02.716989 374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:34929
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:42:02.733292 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34927
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:02.821427 667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:02.821691 667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:02.821722 667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:02.822873 667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:02.822911 667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:02.822937 667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:02.824482 667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34927
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:02.824672 667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:02.824916 667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:02.830552 676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:02.830674 680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:02.830792 667 server_base.cc:1034] running on GCE node
W20260809 08:42:02.830642 674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:02.831103 667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:02.831364 667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:02.832544 667 hybrid_clock.cc:648] HybridClock initialized: now 1786264922832534 us; error 30 us; skew 500 ppm
Aug 09 08:42:02 dist-test-slave-gkw7 krb5kdc[310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264922, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:02.835287 667 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:42:02.835731 667 webserver.cc:460] Webserver started at http://127.31.63.66:45927/ using document root <none> and password file <none>
I20260809 08:42:02.835991 667 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:02.836064 667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:02.836216 667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:02.837384 667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "ad779cf2ece249028f1e1c3d208dcaba"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "7b511f4fbd27d045d4fbf53883dd8096"
server_key_iv: "9944ddd8e93547a48b4ad092690d3007"
server_key_version: "encryptionkey@0"
I20260809 08:42:02.837831 667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "ad779cf2ece249028f1e1c3d208dcaba"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "7b511f4fbd27d045d4fbf53883dd8096"
server_key_iv: "9944ddd8e93547a48b4ad092690d3007"
server_key_version: "encryptionkey@0"
I20260809 08:42:02.839187 667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:02.839956 688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:02.840135 667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:02.840240 667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "ad779cf2ece249028f1e1c3d208dcaba"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "7b511f4fbd27d045d4fbf53883dd8096"
server_key_iv: "9944ddd8e93547a48b4ad092690d3007"
server_key_version: "encryptionkey@0"
I20260809 08:42:02.840323 667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:02.850067 667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:02.852398 667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:02.852571 667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:02.852846 667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:02.853191 667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:02.853230 667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:02.853255 667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:02.853267 667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:02.862195 667 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:35231
I20260809 08:42:02.862315 853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:35231 every 8 connection(s)
I20260809 08:42:02.862635 667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:42:02.868963 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 667
I20260809 08:42:02.869072 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
Aug 09 08:42:02 dist-test-slave-gkw7 krb5kdc[310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264922, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:02.869297 31997 external_mini_cluster.cc:1428] Setting key 517b3565970dfa6ffed1df12a9f7aabc
I20260809 08:42:02.872977 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:02.863462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:33321 (local address 127.31.63.126:34927)
0809 08:42:02.863508 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:02.863510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:02.863735 (+ 225us) server_negotiation.cc:407] Connection header received
0809 08:42:02.864745 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:02.864746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:02.864761 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:02.864814 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:02.865519 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:02.866059 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:02.867115 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:02.867325 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:02.867408 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:02.869935 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:02.869937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:02.869937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:02.869945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:02.871346 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:02.871895 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:02.871896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:02.871897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:02.871969 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:02.872243 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:02.872244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:02.872245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:02.872396 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:02.872469 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:02.872746 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 08:42:02.872842 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:42:02.873008 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:02.863422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:34927 (local address 127.31.63.66:33321)
0809 08:42:02.863580 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:02.863627 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:02.864612 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:02.864903 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:02.864907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:02.865190 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:02.865403 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:02.865408 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:02.866169 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:02.866170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:02.867018 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:02.867020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:02.867417 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:02.867418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:02.867555 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:02.868060 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:02.868069 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:02.869809 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:02.871429 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:02.871437 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:02.871440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:02.871806 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:02.872043 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:02.872044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:02.872045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:02.872173 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:02.872467 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:02.872473 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:02.872601 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 08:42:02.872701 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 08:42:02.872836 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260809 08:42:02.873587 854 heartbeater.cc:347] Connected to a master server at 127.31.63.126:34927
I20260809 08:42:02.873699 854 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:02.873920 854 heartbeater.cc:511] Master 127.31.63.126:34927 requested a full tablet report, sending...
I20260809 08:42:02.874337 374 ts_manager.cc:194] Registered new tserver with Master: ad779cf2ece249028f1e1c3d208dcaba (127.31.63.66:35231)
I20260809 08:42:02.874750 374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:33321
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:42:02.900382 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34927
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:02.986697 864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:02.986912 864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:02.986937 864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:02.988022 864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:02.988059 864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:02.988085 864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:02.989530 864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34927
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:02.989802 864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:02.989984 864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:02.995142 869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:02.995143 873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:02.995216 870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:02.995718 864 server_base.cc:1034] running on GCE node
I20260809 08:42:02.995920 864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:02.996196 864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:02.997397 864 hybrid_clock.cc:648] HybridClock initialized: now 1786264922997377 us; error 36 us; skew 500 ppm
Aug 09 08:42:02 dist-test-slave-gkw7 krb5kdc[310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264922, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:03.000366 864 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:42:03.000859 864 webserver.cc:460] Webserver started at http://127.31.63.67:38631/ using document root <none> and password file <none>
I20260809 08:42:03.001096 864 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:03.001168 864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:03.001327 864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:03.002490 864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "32df3ab0a11e4fecb30971bab179f6b0"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "75bcd81572ff83f35729848d4014c4c1"
server_key_iv: "fa8c46bf754234ff733f23082d8044c8"
server_key_version: "encryptionkey@0"
I20260809 08:42:03.002871 864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "32df3ab0a11e4fecb30971bab179f6b0"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "75bcd81572ff83f35729848d4014c4c1"
server_key_iv: "fa8c46bf754234ff733f23082d8044c8"
server_key_version: "encryptionkey@0"
I20260809 08:42:03.004340 864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:42:03.005167 884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:03.005366 864 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:03.005470 864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "32df3ab0a11e4fecb30971bab179f6b0"
format_stamp: "Formatted at 2026-08-09 08:42:02 on dist-test-slave-gkw7"
server_key: "75bcd81572ff83f35729848d4014c4c1"
server_key_iv: "fa8c46bf754234ff733f23082d8044c8"
server_key_version: "encryptionkey@0"
I20260809 08:42:03.005582 864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:03.022732 864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:03.024837 864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:03.024973 864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:03.025187 864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:03.025660 864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:03.025719 864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:03.025755 864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:03.025784 864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:03.032475 864 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:35137
I20260809 08:42:03.032523 1059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:35137 every 8 connection(s)
I20260809 08:42:03.032868 864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:42:03.036794 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 864
I20260809 08:42:03.036873 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:03.037030 31997 external_mini_cluster.cc:1428] Setting key 5f96f23f58d5a9d97d03aea76a3eeeeb
Aug 09 08:42:03 dist-test-slave-gkw7 krb5kdc[310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264922, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:03.042784 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.033670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:45823 (local address 127.31.63.126:34927)
0809 08:42:03.033742 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:03.033744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:03.033979 (+ 235us) server_negotiation.cc:407] Connection header received
0809 08:42:03.034985 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:03.034987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:03.035001 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:03.035054 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:03.035750 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.036258 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.037474 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.037743 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.037818 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.039977 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:03.039978 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:03.039979 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:03.039987 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:03.041270 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:03.041752 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:03.041753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:03.041754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:03.041826 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:03.042095 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:03.042097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:03.042097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:03.042218 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:03.042294 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:03.042590 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 08:42:03.042681 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:42:03.042966 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.033657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:34927 (local address 127.31.63.67:45823)
0809 08:42:03.033848 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:03.033888 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:03.034862 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:03.035130 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:03.035133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:03.035404 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:03.035605 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.035609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.036362 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.036363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.037390 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.037392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.037842 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.037843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.037979 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.038469 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:03.038477 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:03.039879 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:03.041359 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:03.041368 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:03.041370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:03.041671 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:03.041899 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:03.041900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:03.041901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:03.042022 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:03.042297 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:03.042305 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:03.042448 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 08:42:03.042556 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:42:03.042722 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260809 08:42:03.043479 1060 heartbeater.cc:347] Connected to a master server at 127.31.63.126:34927
I20260809 08:42:03.043577 1060 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:03.043783 1060 heartbeater.cc:511] Master 127.31.63.126:34927 requested a full tablet report, sending...
I20260809 08:42:03.044198 374 ts_manager.cc:194] Registered new tserver with Master: 32df3ab0a11e4fecb30971bab179f6b0 (127.31.63.67:35137)
I20260809 08:42:03.044575 374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:45823
I20260809 08:42:03.048689 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:42:03.056504 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.050629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58920 (local address 127.31.63.126:34927)
0809 08:42:03.050690 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:03.050693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:03.050819 (+ 126us) server_negotiation.cc:407] Connection header received
0809 08:42:03.050858 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:03.050858 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:03.050868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:03.050915 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:03.051442 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.051912 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.052659 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.052852 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.052902 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.054176 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:03.054177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:03.054177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:03.054182 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:03.055478 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:03.055818 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:03.055819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:03.055819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:03.055872 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:03.056040 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:03.056040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:03.056040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:03.056184 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:03.056273 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:03.056375 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 08:42:03.056437 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:42:03.059549 374 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:42:03.061389 374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:03.068876 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.064437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:35137 (local address 127.0.0.1:40052)
0809 08:42:03.064612 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:03.064651 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:03.064724 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:03.065094 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:03.065097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:03.065100 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:03.065175 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.065177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.066396 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.066397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.067595 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.067598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.068538 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.068539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.068619 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.068632 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 08:42:03.068717 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:42:03.068780 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":68,"threads_started":1}
I20260809 08:42:03.069056 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.064447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:33051 (local address 127.0.0.1:53482)
0809 08:42:03.064714 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:03.064724 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:03.064767 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:03.065460 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:03.065461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:03.065463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:03.065503 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.065504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.066843 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.066844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.067910 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.067912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.068819 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.068820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.068885 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.068890 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:42:03.068924 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 08:42:03.068966 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":2,"thread_start_us":108,"threads_started":1}
I20260809 08:42:03.069062 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.064741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.31.63.66:35231)
0809 08:42:03.065238 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:03.065242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:03.065302 (+ 60us) server_negotiation.cc:407] Connection header received
0809 08:42:03.065400 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:03.065403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:03.065418 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:03.065788 (+ 370us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:03.065880 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.066907 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.067983 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.068518 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.068587 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.068644 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:03.068924 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 08:42:03.068972 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":107,"threads_started":1}
I20260809 08:42:03.068886 1077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.064454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:35231 (local address 127.0.0.1:56190)
0809 08:42:03.065173 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:03.065184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:03.065312 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:03.065657 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:03.065659 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:03.065661 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:03.065704 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.065706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.067004 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.067006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.067899 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.067901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.068677 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.068678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.068747 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.068749 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:42:03.068795 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 08:42:03.068825 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":85,"threads_started":1}
I20260809 08:42:03.069486 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.064966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.31.63.65:33051)
0809 08:42:03.065203 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:03.065208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:03.065218 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:03.065263 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:03.065264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:03.065279 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:03.065455 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:03.065617 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.066731 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.067999 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.068719 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.069245 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.069330 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:03.069362 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 08:42:03.069409 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
I20260809 08:42:03.068899 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.064753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40052 (local address 127.31.63.67:35137)
0809 08:42:03.064927 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:03.064932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:03.064940 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:03.064973 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:03.064975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:03.064988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:03.065051 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:03.065321 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.066273 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.067698 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.068436 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.068515 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.068577 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:03.068754 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 08:42:03.068809 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260809 08:42:03.070022 760 tablet_service.cc:1467] Processing CreateTablet for tablet 088e8b10fc6f4a3ebe36928f337c4f9e (DEFAULT_TABLE table=test-table [id=e62227a8ef594dd7aae298b4ab903a07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:03.070188 564 tablet_service.cc:1467] Processing CreateTablet for tablet 088e8b10fc6f4a3ebe36928f337c4f9e (DEFAULT_TABLE table=test-table [id=e62227a8ef594dd7aae298b4ab903a07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:03.070320 952 tablet_service.cc:1467] Processing CreateTablet for tablet 088e8b10fc6f4a3ebe36928f337c4f9e (DEFAULT_TABLE table=test-table [id=e62227a8ef594dd7aae298b4ab903a07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:03.070466 760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 088e8b10fc6f4a3ebe36928f337c4f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:03.070556 564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 088e8b10fc6f4a3ebe36928f337c4f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:03.070618 952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 088e8b10fc6f4a3ebe36928f337c4f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:03.073593 1085 tablet_bootstrap.cc:492] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba: Bootstrap starting.
I20260809 08:42:03.073889 1088 tablet_bootstrap.cc:492] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0: Bootstrap starting.
I20260809 08:42:03.074342 1085 tablet_bootstrap.cc:654] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:03.074621 1088 tablet_bootstrap.cc:654] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:03.075239 1085 log.cc:824] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:03.075321 1088 log.cc:824] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:03.075632 1087 tablet_bootstrap.cc:492] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34: Bootstrap starting.
I20260809 08:42:03.076131 1085 tablet_bootstrap.cc:492] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba: No bootstrap required, opened a new log
I20260809 08:42:03.076140 1088 tablet_bootstrap.cc:492] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0: No bootstrap required, opened a new log
I20260809 08:42:03.076205 1085 ts_tablet_manager.cc:1397] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:42:03.076205 1088 ts_tablet_manager.cc:1397] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:03.076363 1087 tablet_bootstrap.cc:654] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:03.076938 1087 log.cc:824] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:03.077785 1087 tablet_bootstrap.cc:492] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34: No bootstrap required, opened a new log
I20260809 08:42:03.077867 1087 ts_tablet_manager.cc:1397] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:03.078291 1085 raft_consensus.cc:348] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32df3ab0a11e4fecb30971bab179f6b0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 35137 } } peers { permanent_uuid: "a1172095c2964457989e800df407db34" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 33051 } } peers { permanent_uuid: "ad779cf2ece249028f1e1c3d208dcaba" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35231 } }
I20260809 08:42:03.078291 1088 raft_consensus.cc:348] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32df3ab0a11e4fecb30971bab179f6b0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 35137 } } peers { permanent_uuid: "a1172095c2964457989e800df407db34" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 33051 } } peers { permanent_uuid: "ad779cf2ece249028f1e1c3d208dcaba" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35231 } }
I20260809 08:42:03.078481 1085 raft_consensus.cc:374] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:03.078481 1088 raft_consensus.cc:374] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:03.078521 1085 raft_consensus.cc:729] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad779cf2ece249028f1e1c3d208dcaba, State: Initialized, Role: FOLLOWER
I20260809 08:42:03.078521 1088 raft_consensus.cc:729] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32df3ab0a11e4fecb30971bab179f6b0, State: Initialized, Role: FOLLOWER
I20260809 08:42:03.078648 1088 consensus_queue.cc:260] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32df3ab0a11e4fecb30971bab179f6b0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 35137 } } peers { permanent_uuid: "a1172095c2964457989e800df407db34" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 33051 } } peers { permanent_uuid: "ad779cf2ece249028f1e1c3d208dcaba" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35231 } }
I20260809 08:42:03.078712 1085 consensus_queue.cc:260] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32df3ab0a11e4fecb30971bab179f6b0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 35137 } } peers { permanent_uuid: "a1172095c2964457989e800df407db34" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 33051 } } peers { permanent_uuid: "ad779cf2ece249028f1e1c3d208dcaba" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35231 } }
I20260809 08:42:03.078955 1088 ts_tablet_manager.cc:1428] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:42:03.078997 1085 ts_tablet_manager.cc:1428] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:42:03.079161 1060 heartbeater.cc:503] Master 127.31.63.126:34927 was elected leader, sending a full tablet report...
I20260809 08:42:03.079623 854 heartbeater.cc:503] Master 127.31.63.126:34927 was elected leader, sending a full tablet report...
I20260809 08:42:03.079840 1087 raft_consensus.cc:348] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32df3ab0a11e4fecb30971bab179f6b0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 35137 } } peers { permanent_uuid: "a1172095c2964457989e800df407db34" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 33051 } } peers { permanent_uuid: "ad779cf2ece249028f1e1c3d208dcaba" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35231 } }
I20260809 08:42:03.080029 1087 raft_consensus.cc:374] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:03.080092 1087 raft_consensus.cc:729] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1172095c2964457989e800df407db34, State: Initialized, Role: FOLLOWER
I20260809 08:42:03.080224 1087 consensus_queue.cc:260] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32df3ab0a11e4fecb30971bab179f6b0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 35137 } } peers { permanent_uuid: "a1172095c2964457989e800df407db34" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 33051 } } peers { permanent_uuid: "ad779cf2ece249028f1e1c3d208dcaba" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35231 } }
I20260809 08:42:03.080508 1087 ts_tablet_manager.cc:1428] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:42:03.080602 658 heartbeater.cc:503] Master 127.31.63.126:34927 was elected leader, sending a full tablet report...
W20260809 08:42:03.113251 855 tablet.cc:2369] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:42:03.203616 661 tablet.cc:2369] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:42:03.215626 1094 raft_consensus.cc:482] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:03.215745 1094 raft_consensus.cc:504] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32df3ab0a11e4fecb30971bab179f6b0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 35137 } } peers { permanent_uuid: "a1172095c2964457989e800df407db34" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 33051 } } peers { permanent_uuid: "ad779cf2ece249028f1e1c3d208dcaba" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35231 } }
I20260809 08:42:03.216399 1094 leader_election.cc:288] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1172095c2964457989e800df407db34 (127.31.63.65:33051), ad779cf2ece249028f1e1c3d208dcaba (127.31.63.66:35231)
I20260809 08:42:03.219208 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.216549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:33051 (local address 127.31.63.67:41565)
0809 08:42:03.216614 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:03.216626 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:03.216669 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:03.216873 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:03.216874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:03.216875 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:03.216961 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.216963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.217632 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.217633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.218403 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.218405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.218934 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.218935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.219020 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.219023 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:42:03.219092 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:42:03.219131 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":1}
I20260809 08:42:03.219208 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.216672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:41565 (local address 127.31.63.65:33051)
0809 08:42:03.216733 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:03.216735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:03.216744 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:42:03.216775 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:03.216776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:03.216786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:03.216866 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:03.217038 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.217544 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.218498 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.218853 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.218936 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.218945 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:03.219098 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 08:42:03.219135 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:42:03.219496 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.216700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:57419 (local address 127.31.63.66:35231)
0809 08:42:03.216771 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:03.216773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:03.216876 (+ 103us) server_negotiation.cc:407] Connection header received
0809 08:42:03.216958 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:03.216959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:03.216969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:03.217043 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:03.217216 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.217737 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.218690 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.219082 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.219170 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.219178 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:03.219385 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 08:42:03.219432 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:42:03.219712 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.216600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:35231 (local address 127.31.63.67:57419)
0809 08:42:03.216798 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:03.216808 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:03.216891 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:03.217045 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:03.217046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:03.217048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:03.217144 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.217146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.217817 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.217818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.218585 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:03.218587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.219170 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.219171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:03.219278 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.219284 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:42:03.219584 (+ 300us) client_negotiation.cc:240] Negotiation successful
0809 08:42:03.219636 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":116,"threads_started":1}
I20260809 08:42:03.219619 590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088e8b10fc6f4a3ebe36928f337c4f9e" candidate_uuid: "32df3ab0a11e4fecb30971bab179f6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1172095c2964457989e800df407db34" is_pre_election: true
I20260809 08:42:03.219838 590 raft_consensus.cc:2454] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32df3ab0a11e4fecb30971bab179f6b0 in term 0.
I20260809 08:42:03.220041 787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088e8b10fc6f4a3ebe36928f337c4f9e" candidate_uuid: "32df3ab0a11e4fecb30971bab179f6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad779cf2ece249028f1e1c3d208dcaba" is_pre_election: true
I20260809 08:42:03.220155 787 raft_consensus.cc:2454] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32df3ab0a11e4fecb30971bab179f6b0 in term 0.
I20260809 08:42:03.220177 889 leader_election.cc:302] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32df3ab0a11e4fecb30971bab179f6b0, a1172095c2964457989e800df407db34; no voters:
I20260809 08:42:03.220306 1094 raft_consensus.cc:2781] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:03.220387 1094 raft_consensus.cc:482] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:03.220423 1094 raft_consensus.cc:3037] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:03.221216 1094 raft_consensus.cc:504] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32df3ab0a11e4fecb30971bab179f6b0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 35137 } } peers { permanent_uuid: "a1172095c2964457989e800df407db34" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 33051 } } peers { permanent_uuid: "ad779cf2ece249028f1e1c3d208dcaba" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35231 } }
I20260809 08:42:03.221372 1094 leader_election.cc:288] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [CANDIDATE]: Term 1 election: Requested vote from peers a1172095c2964457989e800df407db34 (127.31.63.65:33051), ad779cf2ece249028f1e1c3d208dcaba (127.31.63.66:35231)
I20260809 08:42:03.221522 590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088e8b10fc6f4a3ebe36928f337c4f9e" candidate_uuid: "32df3ab0a11e4fecb30971bab179f6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1172095c2964457989e800df407db34"
I20260809 08:42:03.221617 590 raft_consensus.cc:3037] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:03.221588 787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "088e8b10fc6f4a3ebe36928f337c4f9e" candidate_uuid: "32df3ab0a11e4fecb30971bab179f6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad779cf2ece249028f1e1c3d208dcaba"
I20260809 08:42:03.221688 787 raft_consensus.cc:3037] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:03.222360 590 raft_consensus.cc:2454] T 088e8b10fc6f4a3ebe36928f337c4f9e P a1172095c2964457989e800df407db34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32df3ab0a11e4fecb30971bab179f6b0 in term 1.
I20260809 08:42:03.222558 787 raft_consensus.cc:2454] T 088e8b10fc6f4a3ebe36928f337c4f9e P ad779cf2ece249028f1e1c3d208dcaba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32df3ab0a11e4fecb30971bab179f6b0 in term 1.
I20260809 08:42:03.222568 889 leader_election.cc:302] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32df3ab0a11e4fecb30971bab179f6b0, a1172095c2964457989e800df407db34; no voters:
I20260809 08:42:03.222662 1094 raft_consensus.cc:2781] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:03.222843 1094 raft_consensus.cc:686] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [term 1 LEADER]: Becoming Leader. State: Replica: 32df3ab0a11e4fecb30971bab179f6b0, State: Running, Role: LEADER
I20260809 08:42:03.222955 1094 consensus_queue.cc:237] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32df3ab0a11e4fecb30971bab179f6b0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 35137 } } peers { permanent_uuid: "a1172095c2964457989e800df407db34" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 33051 } } peers { permanent_uuid: "ad779cf2ece249028f1e1c3d208dcaba" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35231 } }
I20260809 08:42:03.223970 373 catalog_manager.cc:5495] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32df3ab0a11e4fecb30971bab179f6b0 (127.31.63.67). New cstate: current_term: 1 leader_uuid: "32df3ab0a11e4fecb30971bab179f6b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32df3ab0a11e4fecb30971bab179f6b0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 35137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1172095c2964457989e800df407db34" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 33051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad779cf2ece249028f1e1c3d208dcaba" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35231 } health_report { overall_health: UNKNOWN } } }
Aug 09 08:42:03 dist-test-slave-gkw7 krb5kdc[310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:42:03.254351 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.246667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53496 (local address 127.31.63.65:33051)
0809 08:42:03.246764 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:03.246765 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:03.246859 (+ 94us) server_negotiation.cc:407] Connection header received
0809 08:42:03.246910 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:03.246911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:03.246920 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:03.246972 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:03.247534 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.248011 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.248689 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.248870 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.248918 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.251358 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:03.251377 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:03.251377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:03.251384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:03.253075 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:03.253454 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:03.253458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:03.253459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:03.253571 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:03.253775 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:03.253776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:03.253776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:03.253937 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:03.254097 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:03.254141 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 08:42:03.254237 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 08:42:03 dist-test-slave-gkw7 krb5kdc[310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264923, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:42:03 dist-test-slave-gkw7 krb5kdc[310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264923, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:42:03.280081 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.271803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.31.63.65:33051)
0809 08:42:03.271863 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:03.271866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:03.271967 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:42:03.272012 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:03.272013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:03.272023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:03.272080 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:03.272781 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.273299 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.274159 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.274381 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.274461 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.276885 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:03.276886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:03.276887 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:03.276894 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:03.278658 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:03.279047 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:03.279048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:03.279048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:03.279114 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:03.279339 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:03.279340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:03.279340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:03.279461 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:03.279554 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:03.279831 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 08:42:03.279947 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 08:42:03.283476 1061 tablet.cc:2369] T 088e8b10fc6f4a3ebe36928f337c4f9e P 32df3ab0a11e4fecb30971bab179f6b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 08:42:03 dist-test-slave-gkw7 krb5kdc[310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:42:03 dist-test-slave-gkw7 krb5kdc[310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:42:03.310346 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:03.303293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53514 (local address 127.31.63.65:33051)
0809 08:42:03.303347 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:03.303349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:03.303433 (+ 84us) server_negotiation.cc:407] Connection header received
0809 08:42:03.303485 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:03.303486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:03.303496 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:03.303546 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:03.304123 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.304594 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.305291 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:03.305467 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:03.305516 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:03.307710 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:03.307711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:03.307712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:03.307718 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:03.309219 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:03.309664 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:03.309665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:03.309665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:03.309721 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:03.309907 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:03.309908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:03.309908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:03.310003 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:03.310076 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:03.310161 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 08:42:03.310237 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 08:42:03.311197 537 tablet_service.cc:2908] Rejecting scan request for tablet 088e8b10fc6f4a3ebe36928f337c4f9e: Uninitialized: clean time has not yet been initialized
I20260809 08:42:03.312330 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 471
I20260809 08:42:03.332132 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 667
I20260809 08:42:03.341955 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 864
I20260809 08:42:03.352649 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 329
2026-08-09T08:42:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2356 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:42:03 dist-test-slave-gkw7 krb5kdc[1124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:42:03 dist-test-slave-gkw7 krb5kdc[1124](info): set up 2 sockets
Aug 09 08:42:03 dist-test-slave-gkw7 krb5kdc[1124](info): commencing operation
krb5kdc: starting...
W20260809 08:42:04.680981 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:42:04 dist-test-slave-gkw7 krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:42:04.806209 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:44469
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:44469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:04.888142 1149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:04.888350 1149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:04.888372 1149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:04.889441 1149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:04.889477 1149 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 08:42:04.889490 1149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:04.889503 1149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:04.889516 1149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:04.889590 1149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:04.891053 1149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:44469
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:44469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:04.891254 1149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:04.891422 1149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:04.896298 1165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:04.896275 1162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:04.896404 1149 server_base.cc:1034] running on GCE node
W20260809 08:42:04.896497 1159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:04.896715 1149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:04.897015 1149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:04.898183 1149 hybrid_clock.cc:648] HybridClock initialized: now 1786264924898158 us; error 41 us; skew 500 ppm
Aug 09 08:42:04 dist-test-slave-gkw7 krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264924, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:04.900632 1149 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:42:04.901080 1149 webserver.cc:460] Webserver started at http://127.31.63.126:44953/ using document root <none> and password file <none>
I20260809 08:42:04.901327 1149 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:04.901405 1149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:04.901607 1149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:04.902688 1149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "7be99251fc05410cb678240287118409"
format_stamp: "Formatted at 2026-08-09 08:42:04 on dist-test-slave-gkw7"
server_key: "0c26298e6845d1092e343d2bd634fe79"
server_key_iv: "8e2f7179557ddccca119120c57df54cb"
server_key_version: "encryptionkey@0"
I20260809 08:42:04.903054 1149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "7be99251fc05410cb678240287118409"
format_stamp: "Formatted at 2026-08-09 08:42:04 on dist-test-slave-gkw7"
server_key: "0c26298e6845d1092e343d2bd634fe79"
server_key_iv: "8e2f7179557ddccca119120c57df54cb"
server_key_version: "encryptionkey@0"
I20260809 08:42:04.904395 1149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:04.905107 1174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:04.905285 1149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:04.905390 1149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "7be99251fc05410cb678240287118409"
format_stamp: "Formatted at 2026-08-09 08:42:04 on dist-test-slave-gkw7"
server_key: "0c26298e6845d1092e343d2bd634fe79"
server_key_iv: "8e2f7179557ddccca119120c57df54cb"
server_key_version: "encryptionkey@0"
I20260809 08:42:04.905476 1149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:04.922678 1149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:04.924461 1149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:04.924628 1149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:04.929682 1149 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:44469
I20260809 08:42:04.929752 1275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:44469 every 8 connection(s)
I20260809 08:42:04.930066 1149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:04.930716 1276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:04.931119 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 1149
I20260809 08:42:04.931196 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:04.931389 31997 external_mini_cluster.cc:1428] Setting key 260c03a4426ffb23041e1701fc1ed453
I20260809 08:42:04.933816 1276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409: Bootstrap starting.
I20260809 08:42:04.934552 1276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:04.934840 1276 log.cc:824] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:04.935582 1276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409: No bootstrap required, opened a new log
I20260809 08:42:04.938019 1276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7be99251fc05410cb678240287118409" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44469 } }
I20260809 08:42:04.938210 1276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:04.938246 1276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be99251fc05410cb678240287118409, State: Initialized, Role: FOLLOWER
I20260809 08:42:04.938385 1276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7be99251fc05410cb678240287118409" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44469 } }
I20260809 08:42:04.938470 1276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:04.938503 1276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:04.938550 1276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 08:42:04 dist-test-slave-gkw7 krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:04.939162 1276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7be99251fc05410cb678240287118409" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44469 } }
I20260809 08:42:04.939287 1276 leader_election.cc:302] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7be99251fc05410cb678240287118409; no voters:
I20260809 08:42:04.939486 1276 leader_election.cc:288] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:04.939540 1286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:04.939765 1276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:04.939770 1286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [term 1 LEADER]: Becoming Leader. State: Replica: 7be99251fc05410cb678240287118409, State: Running, Role: LEADER
I20260809 08:42:04.939921 1286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7be99251fc05410cb678240287118409" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44469 } }
I20260809 08:42:04.940271 1290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7be99251fc05410cb678240287118409. Latest consensus state: current_term: 1 leader_uuid: "7be99251fc05410cb678240287118409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7be99251fc05410cb678240287118409" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44469 } } }
I20260809 08:42:04.940361 1288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7be99251fc05410cb678240287118409" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7be99251fc05410cb678240287118409" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44469 } } }
I20260809 08:42:04.940469 1288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:04.940433 1290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:04.940732 1297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:04.941256 1297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:04.943058 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:04.931950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46478 (local address 127.31.63.126:44469)
0809 08:42:04.932145 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:04.932156 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:04.932190 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:42:04.932746 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:04.932754 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:04.932814 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:04.932966 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:04.933580 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:04.934926 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:04.935825 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:04.936422 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:04.936545 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:04.939066 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:04.939070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:04.939070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:04.939082 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:04.941350 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:04.941829 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:04.941832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:04.941833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:04.941950 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:04.942217 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:04.942218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:04.942219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:04.942468 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:04.942632 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:04.942762 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 08:42:04.942902 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260809 08:42:04.946507 1297 catalog_manager.cc:1347] Generated new cluster ID: 685078298e94406eb9f7e500eeda3d55
I20260809 08:42:04.946569 1297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:04.954592 1297 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:04.955101 1297 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:04.960983 1297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409: Generated new TSK 0
I20260809 08:42:04.961171 1297 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:42:04.994833 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:44469
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:05.097590 1319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:05.097831 1319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:05.097877 1319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:05.099073 1319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:05.099141 1319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:05.099180 1319 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:42:05.099243 1319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:05.100771 1319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.63.126:44469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:05.101028 1319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:05.101274 1319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:05.106482 1328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:05.106529 1330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:05.106508 1334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:05.106773 1319 server_base.cc:1034] running on GCE node
I20260809 08:42:05.107040 1319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:05.107309 1319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:05.108477 1319 hybrid_clock.cc:648] HybridClock initialized: now 1786264925108456 us; error 38 us; skew 500 ppm
Aug 09 08:42:05 dist-test-slave-gkw7 krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264925, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:05.111127 1319 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:42:05.111618 1319 webserver.cc:460] Webserver started at http://127.31.63.65:44761/ using document root <none> and password file <none>
I20260809 08:42:05.111881 1319 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:05.111953 1319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:05.112103 1319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:05.113170 1319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "1485ee498ee94f95a1ae256b8d276682"
format_stamp: "Formatted at 2026-08-09 08:42:05 on dist-test-slave-gkw7"
server_key: "d635f7a8d5d1369d0d2133046c805dc5"
server_key_iv: "d0ef58500aa0ec1e68bc7a14aa16d4c0"
server_key_version: "encryptionkey@0"
I20260809 08:42:05.113597 1319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "1485ee498ee94f95a1ae256b8d276682"
format_stamp: "Formatted at 2026-08-09 08:42:05 on dist-test-slave-gkw7"
server_key: "d635f7a8d5d1369d0d2133046c805dc5"
server_key_iv: "d0ef58500aa0ec1e68bc7a14aa16d4c0"
server_key_version: "encryptionkey@0"
I20260809 08:42:05.114976 1319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:05.115729 1347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:05.115909 1319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:42:05.116014 1319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "1485ee498ee94f95a1ae256b8d276682"
format_stamp: "Formatted at 2026-08-09 08:42:05 on dist-test-slave-gkw7"
server_key: "d635f7a8d5d1369d0d2133046c805dc5"
server_key_iv: "d0ef58500aa0ec1e68bc7a14aa16d4c0"
server_key_version: "encryptionkey@0"
I20260809 08:42:05.116099 1319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:05.125360 1319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:05.127261 1319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:05.127430 1319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:05.128031 1319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:05.128094 1319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:05.128137 1319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:05.128165 1319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:42:05 dist-test-slave-gkw7 krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264925, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:05.136401 1319 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:34369
I20260809 08:42:05.136471 1509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:34369 every 8 connection(s)
I20260809 08:42:05.136850 1319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:05.138949 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.128551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:33801 (local address 127.31.63.126:44469)
0809 08:42:05.128595 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.128598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.128817 (+ 219us) server_negotiation.cc:407] Connection header received
0809 08:42:05.129340 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.129341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.129355 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.129433 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:05.130290 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.131171 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.132307 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.132533 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.132618 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.135996 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:05.135998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:05.135998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:05.136007 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:05.137792 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.138151 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.138152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.138152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.138205 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.138444 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.138444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.138445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.138551 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:05.138601 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.138810 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.138884 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:42:05.139156 1362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.128535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:44469 (local address 127.31.63.65:33801)
0809 08:42:05.128699 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.128736 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.129232 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.129504 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.129507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.129848 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:05.130170 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.130177 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.131293 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.131294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.132197 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.132200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.132632 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.132633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.132765 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.133374 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:05.133385 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:05.135893 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:05.137864 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.137869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.137871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.138084 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.138288 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.138289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.138289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.138388 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.138606 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:05.138610 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:05.138720 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.138799 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.138917 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":97,"threads_started":1}
I20260809 08:42:05.139689 1512 heartbeater.cc:347] Connected to a master server at 127.31.63.126:44469
I20260809 08:42:05.139799 1512 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:05.139988 1512 heartbeater.cc:511] Master 127.31.63.126:44469 requested a full tablet report, sending...
I20260809 08:42:05.140605 1202 ts_manager.cc:194] Registered new tserver with Master: 1485ee498ee94f95a1ae256b8d276682 (127.31.63.65:34369)
I20260809 08:42:05.141166 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 1319
I20260809 08:42:05.141247 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:05.141357 1202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:33801
I20260809 08:42:05.141430 31997 external_mini_cluster.cc:1428] Setting key fc1fdd82fffb1cb7270b192e46aa77ef
I20260809 08:42:05.147617 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.140807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:39737 (local address 127.31.63.126:44469)
0809 08:42:05.140845 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.140847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.141688 (+ 841us) server_negotiation.cc:407] Connection header received
0809 08:42:05.141755 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.141756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.141766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.141830 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:05.142240 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.142738 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.143826 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.144137 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.144208 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.145116 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:05.145117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:05.145117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:05.145123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:05.146498 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.146936 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.146936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.146937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.146984 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.147189 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.147190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.147190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.147282 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:05.147331 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.147498 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.147558 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:42:05.147775 1522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.140754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:44469 (local address 127.31.63.65:39737)
0809 08:42:05.141636 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.141648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.141702 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.141851 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.141851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.142075 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:05.142151 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.142153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.142846 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.142847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.143715 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.143718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.144214 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.144223 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.144285 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.144751 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:05.144757 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:05.145033 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:05.146586 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.146588 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.146588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.146851 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.147050 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.147051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.147051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.147123 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.147348 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:05.147349 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:05.147403 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.147484 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.147602 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:42:05.169437 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:44469
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:05.254716 1525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:05.254977 1525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:05.255020 1525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:05.256210 1525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:05.256294 1525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:05.256326 1525 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:42:05.256364 1525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:05.257889 1525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.63.126:44469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:05.258169 1525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:05.258430 1525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:05.263837 1535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:05.263844 1533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:05.263844 1532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:05.264155 1525 server_base.cc:1034] running on GCE node
I20260809 08:42:05.264398 1525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:05.264674 1525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:05.265844 1525 hybrid_clock.cc:648] HybridClock initialized: now 1786264925265821 us; error 38 us; skew 500 ppm
Aug 09 08:42:05 dist-test-slave-gkw7 krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264925, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:05.268460 1525 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:42:05.268930 1525 webserver.cc:460] Webserver started at http://127.31.63.66:39605/ using document root <none> and password file <none>
I20260809 08:42:05.269166 1525 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:05.269212 1525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:05.269366 1525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:05.270519 1525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "4ee53ae8640c4b788191e4e7ca4451e5"
format_stamp: "Formatted at 2026-08-09 08:42:05 on dist-test-slave-gkw7"
server_key: "0f12a32042a7448c493b5aafa23799d4"
server_key_iv: "75e573f28e6b8d13e64c479d7b33e679"
server_key_version: "encryptionkey@0"
I20260809 08:42:05.270895 1525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "4ee53ae8640c4b788191e4e7ca4451e5"
format_stamp: "Formatted at 2026-08-09 08:42:05 on dist-test-slave-gkw7"
server_key: "0f12a32042a7448c493b5aafa23799d4"
server_key_iv: "75e573f28e6b8d13e64c479d7b33e679"
server_key_version: "encryptionkey@0"
I20260809 08:42:05.272344 1525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:05.273100 1544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:05.273437 1525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:05.273564 1525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "4ee53ae8640c4b788191e4e7ca4451e5"
format_stamp: "Formatted at 2026-08-09 08:42:05 on dist-test-slave-gkw7"
server_key: "0f12a32042a7448c493b5aafa23799d4"
server_key_iv: "75e573f28e6b8d13e64c479d7b33e679"
server_key_version: "encryptionkey@0"
I20260809 08:42:05.273665 1525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:05.291005 1525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:05.293074 1525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:05.293246 1525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:05.294009 1525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:05.294083 1525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:05.294121 1525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:05.294143 1525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:42:05 dist-test-slave-gkw7 krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264925, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:05.302249 1525 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:33663
I20260809 08:42:05.302335 1706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:33663 every 8 connection(s)
I20260809 08:42:05.302675 1525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:42:05.304000 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.294507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:39443 (local address 127.31.63.126:44469)
0809 08:42:05.294586 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.294588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.294846 (+ 258us) server_negotiation.cc:407] Connection header received
0809 08:42:05.295354 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.295355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.295370 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.295452 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:05.296101 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.296955 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.298165 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.298390 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.298481 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.300962 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:05.300964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:05.300964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:05.300973 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:05.302742 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.303287 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.303293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.303293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.303354 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.303539 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.303539 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.303539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.303639 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:05.303686 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.303874 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.303936 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:42:05.304284 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.294492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:44469 (local address 127.31.63.66:39443)
0809 08:42:05.294703 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.294746 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.295254 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.295506 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.295510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.295757 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:05.296021 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.296027 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.297060 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.297061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.298044 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.298047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.298495 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.298496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.298640 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.299145 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:05.299154 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:05.300857 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:05.302844 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.302851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.302853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.303198 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.303405 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.303405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.303406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.303502 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.303675 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:05.303678 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:05.303788 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.303864 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.303999 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":137,"threads_started":1}
I20260809 08:42:05.304860 1708 heartbeater.cc:347] Connected to a master server at 127.31.63.126:44469
I20260809 08:42:05.304970 1708 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:05.305190 1708 heartbeater.cc:511] Master 127.31.63.126:44469 requested a full tablet report, sending...
I20260809 08:42:05.305322 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 1525
I20260809 08:42:05.305408 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:42:05.305634 31997 external_mini_cluster.cc:1428] Setting key 2538890a688d6ea663117085881db3fe
I20260809 08:42:05.305651 1202 ts_manager.cc:194] Registered new tserver with Master: 4ee53ae8640c4b788191e4e7ca4451e5 (127.31.63.66:33663)
I20260809 08:42:05.306268 1202 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:39443
I20260809 08:42:05.312777 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.306225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:36179 (local address 127.31.63.126:44469)
0809 08:42:05.306341 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.306343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.306351 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:05.306370 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.306371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.306381 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.306432 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:05.307027 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.307617 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.308676 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.308986 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.309131 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.310215 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:05.310216 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:05.310216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:05.310222 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:05.311577 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.312024 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.312025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.312025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.312075 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.312286 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.312286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.312286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.312411 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:05.312460 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.312651 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.312714 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 08:42:05.313071 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.306032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:44469 (local address 127.31.63.66:36179)
0809 08:42:05.306190 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.306203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.306260 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.306607 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.306608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.306848 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:05.306931 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.306933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.307711 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.307713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.308592 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.308595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.309099 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.309100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.309188 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.309680 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:05.309684 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:05.310118 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:05.311693 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.311694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.311695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.311948 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.312147 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.312148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.312149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.312223 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.312477 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:05.312478 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:05.312536 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.312635 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.312763 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:42:05.336519 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:44469
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:05.423574 1723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:05.423786 1723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:05.423813 1723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:05.424928 1723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:05.424963 1723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:05.424984 1723 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:42:05.425001 1723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:05.426523 1723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.63.126:44469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:05.426718 1723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:05.426900 1723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:05.432139 1731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:05.432328 1723 server_base.cc:1034] running on GCE node
W20260809 08:42:05.432205 1734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:05.432139 1736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:05.432691 1723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:05.432986 1723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:05.434145 1723 hybrid_clock.cc:648] HybridClock initialized: now 1786264925434113 us; error 45 us; skew 500 ppm
Aug 09 08:42:05 dist-test-slave-gkw7 krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264925, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:05.436856 1723 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:42:05.437377 1723 webserver.cc:460] Webserver started at http://127.31.63.67:42531/ using document root <none> and password file <none>
I20260809 08:42:05.437703 1723 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:05.437776 1723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:05.437940 1723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:05.439038 1723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "63ce77d13705406fa5e8973766dd6294"
format_stamp: "Formatted at 2026-08-09 08:42:05 on dist-test-slave-gkw7"
server_key: "78719e1fdd8d030ba956a639db8af259"
server_key_iv: "3429d5cacf2f11a1cf19d03e5b868428"
server_key_version: "encryptionkey@0"
I20260809 08:42:05.439427 1723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "63ce77d13705406fa5e8973766dd6294"
format_stamp: "Formatted at 2026-08-09 08:42:05 on dist-test-slave-gkw7"
server_key: "78719e1fdd8d030ba956a639db8af259"
server_key_iv: "3429d5cacf2f11a1cf19d03e5b868428"
server_key_version: "encryptionkey@0"
I20260809 08:42:05.441061 1723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:42:05.441953 1745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:05.442144 1723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:42:05.442248 1723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "63ce77d13705406fa5e8973766dd6294"
format_stamp: "Formatted at 2026-08-09 08:42:05 on dist-test-slave-gkw7"
server_key: "78719e1fdd8d030ba956a639db8af259"
server_key_iv: "3429d5cacf2f11a1cf19d03e5b868428"
server_key_version: "encryptionkey@0"
I20260809 08:42:05.442344 1723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:05.460767 1723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:05.463212 1723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:05.463392 1723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:05.464020 1723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:05.464080 1723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:05.464126 1723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:05.464159 1723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:42:05 dist-test-slave-gkw7 krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264925, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:05.472721 1723 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:46607
I20260809 08:42:05.472818 1909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:46607 every 8 connection(s)
I20260809 08:42:05.473191 1723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:42:05.474121 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.464410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:56817 (local address 127.31.63.126:44469)
0809 08:42:05.464479 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.464482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.464745 (+ 263us) server_negotiation.cc:407] Connection header received
0809 08:42:05.465304 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.465305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.465317 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.465375 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:05.466150 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.466684 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.467743 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.467948 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.468027 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.470515 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:05.470516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:05.470517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:05.470525 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:05.472380 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.472902 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.472904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.472905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.472971 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.473291 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.473292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.473293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.473479 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:05.473577 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.473978 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.474057 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:42:05.474248 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.464422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:44469 (local address 127.31.63.67:56817)
0809 08:42:05.464589 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.464632 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.465195 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.465454 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.465458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.465843 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:05.466042 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.466047 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.466786 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.466788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.467625 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.467627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.468038 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.468039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.468166 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.468730 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:05.468738 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:05.470412 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:05.472503 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.472510 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.472513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.472825 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.473043 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.473043 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.473044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.473192 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.473672 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:05.473676 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:05.473833 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.473925 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.474051 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260809 08:42:05.474960 1911 heartbeater.cc:347] Connected to a master server at 127.31.63.126:44469
I20260809 08:42:05.475072 1911 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:05.475263 1911 heartbeater.cc:511] Master 127.31.63.126:44469 requested a full tablet report, sending...
I20260809 08:42:05.475718 1200 ts_manager.cc:194] Registered new tserver with Master: 63ce77d13705406fa5e8973766dd6294 (127.31.63.67:46607)
I20260809 08:42:05.476157 1200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:56817
I20260809 08:42:05.481628 1917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.476137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:44469 (local address 127.31.63.67:39453)
0809 08:42:05.476356 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.476367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.476426 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.476538 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.476539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.476742 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:05.476807 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.476809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.477640 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.477641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.478235 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.478237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.478548 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.478549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.478612 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.479079 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:05.479084 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:05.479380 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:05.480665 (+ 1285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.480666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.480666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.480833 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.481068 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.481069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.481069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.481123 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.481334 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:05.481334 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:05.481372 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.481423 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.481503 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":154,"threads_started":1}
I20260809 08:42:05.481634 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.476252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:39453 (local address 127.31.63.126:44469)
0809 08:42:05.476307 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.476308 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.476413 (+ 105us) server_negotiation.cc:407] Connection header received
0809 08:42:05.476480 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.476481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.476488 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.476533 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:05.476882 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.477506 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.478305 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.478482 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.478533 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.479452 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:05.479453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:05.479453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:05.479458 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:05.480591 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.480944 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.480945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.480945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.481007 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.481174 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.481175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.481175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.481277 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:05.481328 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.481442 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.481496 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:42:05.482822 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 1723
I20260809 08:42:05.482924 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:05.483139 31997 external_mini_cluster.cc:1428] Setting key 525bb435f7a72921837c8c13f1a0d873
I20260809 08:42:05.483989 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:42:05 dist-test-slave-gkw7 krb5kdc[1124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264925, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:42:05 dist-test-slave-gkw7 krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264925, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:05.510017 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.502829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.31.63.126:44469)
0809 08:42:05.502902 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.502904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.502914 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:05.502982 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.502983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.502993 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.503047 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:05.503714 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.504264 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.505020 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.505203 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.505255 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.507549 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:05.507550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:05.507551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:05.507557 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:05.508927 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.509290 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.509291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.509291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.509352 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.509509 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.509510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.509510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.509675 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:05.509761 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.509847 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.509909 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 08:42:05.511775 1200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:42:05.513414 1200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:05.520452 1934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.515848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:46607 (local address 127.0.0.1:34788)
0809 08:42:05.516194 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.516203 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.516246 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.516638 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.516639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.516641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:05.516686 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.516687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.517997 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.517998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.519136 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.519138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.520144 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.520144 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.520204 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.520216 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.520313 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.520367 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":52,"thread_start_us":88,"threads_started":1}
I20260809 08:42:05.520493 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.516063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34788 (local address 127.31.63.67:46607)
0809 08:42:05.516402 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.516407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.516416 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:42:05.516460 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.516462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.516474 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.516571 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:05.516784 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.517912 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.519243 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.520056 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.520137 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.520212 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.520341 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.520389 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":153,"threads_started":1}
I20260809 08:42:05.520540 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.515848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:34369 (local address 127.0.0.1:46348)
0809 08:42:05.516027 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.516067 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.516134 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.516531 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.516534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.516536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:05.516608 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.516610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.518015 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.518016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.519388 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.519391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.520288 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.520289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.520347 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.520350 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.520399 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.520432 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":143,"thread_start_us":69,"threads_started":1}
I20260809 08:42:05.520839 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.515972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46348 (local address 127.31.63.65:34369)
0809 08:42:05.516330 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.516335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.516345 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:05.516403 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.516405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.516423 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.516513 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:05.516705 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.517896 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.519483 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.520199 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.520272 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.520342 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.520677 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.520731 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":96,"threads_started":1}
I20260809 08:42:05.521261 1815 tablet_service.cc:1467] Processing CreateTablet for tablet c9490e968baf47e280707b2b6ab6dd51 (DEFAULT_TABLE table=test-table [id=addb9879574846ab998e7fe52348ba51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:05.521387 1417 tablet_service.cc:1467] Processing CreateTablet for tablet c9490e968baf47e280707b2b6ab6dd51 (DEFAULT_TABLE table=test-table [id=addb9879574846ab998e7fe52348ba51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:05.521731 1417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9490e968baf47e280707b2b6ab6dd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:05.521744 1815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9490e968baf47e280707b2b6ab6dd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:05.521988 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.516066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:33663 (local address 127.0.0.1:43406)
0809 08:42:05.516211 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.516224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.516284 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.516856 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.516856 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.516859 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:05.516930 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.516932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.517966 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.517967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.519388 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.519391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.521712 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.521713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.521800 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.521804 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.521869 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.521910 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":70,"thread_start_us":78,"threads_started":1}
I20260809 08:42:05.522029 1941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.516429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43406 (local address 127.31.63.66:33663)
0809 08:42:05.516669 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.516673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.516683 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:05.516734 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.516736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.516751 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.517009 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:05.517057 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.517871 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.520383 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.521610 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.521712 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.521772 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.521884 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.521923 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":102,"threads_started":1}
I20260809 08:42:05.525308 1942 tablet_bootstrap.cc:492] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: Bootstrap starting.
I20260809 08:42:05.525797 1943 tablet_bootstrap.cc:492] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: Bootstrap starting.
I20260809 08:42:05.525902 1615 tablet_service.cc:1467] Processing CreateTablet for tablet c9490e968baf47e280707b2b6ab6dd51 (DEFAULT_TABLE table=test-table [id=addb9879574846ab998e7fe52348ba51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:05.526292 1615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9490e968baf47e280707b2b6ab6dd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:05.526405 1942 tablet_bootstrap.cc:654] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:05.526470 1943 tablet_bootstrap.cc:654] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:05.526734 1942 log.cc:824] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:05.526822 1943 log.cc:824] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:05.527568 1942 tablet_bootstrap.cc:492] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: No bootstrap required, opened a new log
I20260809 08:42:05.527652 1942 ts_tablet_manager.cc:1397] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:05.529062 1942 raft_consensus.cc:348] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.529237 1942 raft_consensus.cc:374] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:05.529284 1942 raft_consensus.cc:729] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63ce77d13705406fa5e8973766dd6294, State: Initialized, Role: FOLLOWER
I20260809 08:42:05.529404 1942 consensus_queue.cc:260] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.529606 1949 tablet_bootstrap.cc:492] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: Bootstrap starting.
I20260809 08:42:05.529750 1942 ts_tablet_manager.cc:1428] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:42:05.529913 1911 heartbeater.cc:503] Master 127.31.63.126:44469 was elected leader, sending a full tablet report...
I20260809 08:42:05.530314 1949 tablet_bootstrap.cc:654] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:05.530440 1943 tablet_bootstrap.cc:492] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: No bootstrap required, opened a new log
I20260809 08:42:05.530539 1943 ts_tablet_manager.cc:1397] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 08:42:05.530681 1949 log.cc:824] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:05.531674 1949 tablet_bootstrap.cc:492] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: No bootstrap required, opened a new log
I20260809 08:42:05.531759 1949 ts_tablet_manager.cc:1397] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:05.532444 1943 raft_consensus.cc:348] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.532601 1943 raft_consensus.cc:374] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:05.532649 1943 raft_consensus.cc:729] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1485ee498ee94f95a1ae256b8d276682, State: Initialized, Role: FOLLOWER
I20260809 08:42:05.532778 1943 consensus_queue.cc:260] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.533036 1943 ts_tablet_manager.cc:1428] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:42:05.533126 1512 heartbeater.cc:503] Master 127.31.63.126:44469 was elected leader, sending a full tablet report...
I20260809 08:42:05.533432 1949 raft_consensus.cc:348] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.533640 1949 raft_consensus.cc:374] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:05.533690 1949 raft_consensus.cc:729] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee53ae8640c4b788191e4e7ca4451e5, State: Initialized, Role: FOLLOWER
I20260809 08:42:05.533818 1949 consensus_queue.cc:260] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.534108 1949 ts_tablet_manager.cc:1428] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:42:05.534250 1708 heartbeater.cc:503] Master 127.31.63.126:44469 was elected leader, sending a full tablet report...
W20260809 08:42:05.553328 1709 tablet.cc:2369] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:42:05.604869 1951 raft_consensus.cc:482] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:05.605024 1951 raft_consensus.cc:504] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.605607 1951 leader_election.cc:288] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1485ee498ee94f95a1ae256b8d276682 (127.31.63.65:34369), 4ee53ae8640c4b788191e4e7ca4451e5 (127.31.63.66:33663)
I20260809 08:42:05.609292 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.605790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:34369 (local address 127.31.63.67:49119)
0809 08:42:05.605853 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.605867 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.605922 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.606155 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.606156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.606158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:05.606248 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.606250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.607265 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.607265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.608277 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.608280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.608955 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.608956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.609054 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.609060 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.609123 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.609171 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 08:42:05.609311 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.605907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:49119 (local address 127.31.63.65:34369)
0809 08:42:05.605980 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.605983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.605996 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:42:05.606030 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.606031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.606053 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.606129 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:05.606336 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.607173 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.608369 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.608865 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.608940 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.608949 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.609153 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.609220 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:42:05.609483 1941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.605978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:51721 (local address 127.31.63.66:33663)
0809 08:42:05.606022 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.606024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.606206 (+ 182us) server_negotiation.cc:407] Connection header received
0809 08:42:05.606322 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.606323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.606335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.606399 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:05.606556 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.607298 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.608581 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.609055 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.609133 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.609142 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.609339 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.609394 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:42:05.609692 1446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9490e968baf47e280707b2b6ab6dd51" candidate_uuid: "63ce77d13705406fa5e8973766dd6294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1485ee498ee94f95a1ae256b8d276682" is_pre_election: true
I20260809 08:42:05.609819 1956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.605889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:33663 (local address 127.31.63.67:51721)
0809 08:42:05.606135 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.606146 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.606214 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.606410 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.606411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.606412 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:05.606479 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.606480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.607427 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.607434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.608483 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.608486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.609160 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.609161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.609246 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.609250 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.609317 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.609745 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":1,"thread_start_us":144,"threads_started":1}
I20260809 08:42:05.609870 1446 raft_consensus.cc:2454] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63ce77d13705406fa5e8973766dd6294 in term 0.
I20260809 08:42:05.610176 1747 leader_election.cc:302] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1485ee498ee94f95a1ae256b8d276682, 63ce77d13705406fa5e8973766dd6294; no voters:
I20260809 08:42:05.610275 1644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9490e968baf47e280707b2b6ab6dd51" candidate_uuid: "63ce77d13705406fa5e8973766dd6294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" is_pre_election: true
I20260809 08:42:05.610409 1951 raft_consensus.cc:2781] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:05.610486 1951 raft_consensus.cc:482] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:05.610474 1644 raft_consensus.cc:2454] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63ce77d13705406fa5e8973766dd6294 in term 0.
I20260809 08:42:05.610523 1951 raft_consensus.cc:3037] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:05.611387 1951 raft_consensus.cc:504] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.611546 1951 leader_election.cc:288] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [CANDIDATE]: Term 1 election: Requested vote from peers 1485ee498ee94f95a1ae256b8d276682 (127.31.63.65:34369), 4ee53ae8640c4b788191e4e7ca4451e5 (127.31.63.66:33663)
I20260809 08:42:05.611670 1446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9490e968baf47e280707b2b6ab6dd51" candidate_uuid: "63ce77d13705406fa5e8973766dd6294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1485ee498ee94f95a1ae256b8d276682"
I20260809 08:42:05.611765 1644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9490e968baf47e280707b2b6ab6dd51" candidate_uuid: "63ce77d13705406fa5e8973766dd6294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee53ae8640c4b788191e4e7ca4451e5"
I20260809 08:42:05.611763 1446 raft_consensus.cc:3037] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:05.611845 1644 raft_consensus.cc:3037] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:05.612785 1644 raft_consensus.cc:2454] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63ce77d13705406fa5e8973766dd6294 in term 1.
I20260809 08:42:05.612785 1446 raft_consensus.cc:2454] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63ce77d13705406fa5e8973766dd6294 in term 1.
I20260809 08:42:05.613595 1748 leader_election.cc:302] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee53ae8640c4b788191e4e7ca4451e5, 63ce77d13705406fa5e8973766dd6294; no voters:
I20260809 08:42:05.613744 1951 raft_consensus.cc:2781] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:05.613936 1951 raft_consensus.cc:686] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 1 LEADER]: Becoming Leader. State: Replica: 63ce77d13705406fa5e8973766dd6294, State: Running, Role: LEADER
I20260809 08:42:05.614046 1951 consensus_queue.cc:237] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.614912 1200 catalog_manager.cc:5495] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 63ce77d13705406fa5e8973766dd6294 (127.31.63.67). New cstate: current_term: 1 leader_uuid: "63ce77d13705406fa5e8973766dd6294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } health_report { overall_health: UNKNOWN } } }
I20260809 08:42:05.637305 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.634852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34792 (local address 127.31.63.67:46607)
0809 08:42:05.634915 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.634917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.634925 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:05.635005 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.635006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.635024 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.635087 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:05.635209 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.635687 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.636436 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.636639 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.636695 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.636879 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:05.636973 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:05.637031 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.637080 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.637114 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 08:42:05.637748 1513 tablet.cc:2369] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:42:05.639526 1644 raft_consensus.cc:1261] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 FOLLOWER]: Refusing update from remote peer 63ce77d13705406fa5e8973766dd6294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:05.639577 1446 raft_consensus.cc:1261] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682 [term 1 FOLLOWER]: Refusing update from remote peer 63ce77d13705406fa5e8973766dd6294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:05.639966 1951 consensus_queue.cc:1035] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:05.640110 1959 consensus_queue.cc:1035] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:05.644234 1964 mvcc.cc:205] Tried to move back new op lower bound from 7316541135415877632 to 7316541135315943424. Current Snapshot: MvccSnapshot[applied={T|T < 7316541135415877632}]
I20260809 08:42:05.644407 1965 mvcc.cc:205] Tried to move back new op lower bound from 7316541135415877632 to 7316541135315943424. Current Snapshot: MvccSnapshot[applied={T|T < 7316541135415877632}]
I20260809 08:42:05.649117 1199 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:05.649273 1199 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:05.650296 1199 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409: Sending DeleteTablet for 3 replicas of tablet c9490e968baf47e280707b2b6ab6dd51
I20260809 08:42:05.650764 1815 tablet_service.cc:1514] Processing DeleteTablet for tablet c9490e968baf47e280707b2b6ab6dd51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:05 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:34788
I20260809 08:42:05.650759 1417 tablet_service.cc:1514] Processing DeleteTablet for tablet c9490e968baf47e280707b2b6ab6dd51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:05 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:46348
I20260809 08:42:05.651167 1979 tablet_replica.cc:330] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: stopping tablet replica
I20260809 08:42:05.651324 1979 raft_consensus.cc:2229] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:05.651298 1199 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:42:05.651446 1199 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:05.651796 1979 raft_consensus.cc:2258] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:05.652737 1614 tablet_service.cc:1467] Processing CreateTablet for tablet 0e788790f47147588ea1aba54dedfc8f (DEFAULT_TABLE table=test-table [id=cef5cd23cf654e3384a9ffb3a87759a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:05.652798 1815 tablet_service.cc:1467] Processing CreateTablet for tablet 0e788790f47147588ea1aba54dedfc8f (DEFAULT_TABLE table=test-table [id=cef5cd23cf654e3384a9ffb3a87759a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:05.652855 1615 tablet_service.cc:1514] Processing DeleteTablet for tablet c9490e968baf47e280707b2b6ab6dd51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:05 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:43406
I20260809 08:42:05.652899 1815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e788790f47147588ea1aba54dedfc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:05.652900 1614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e788790f47147588ea1aba54dedfc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:05.653091 1980 tablet_replica.cc:330] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: stopping tablet replica
I20260809 08:42:05.653184 1980 raft_consensus.cc:2229] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:42:05.653301 1980 raft_consensus.cc:2258] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:05.653621 1417 tablet_service.cc:1467] Processing CreateTablet for tablet 0e788790f47147588ea1aba54dedfc8f (DEFAULT_TABLE table=test-table [id=cef5cd23cf654e3384a9ffb3a87759a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:05.653766 1417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e788790f47147588ea1aba54dedfc8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:05.653812 1980 ts_tablet_manager.cc:1905] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:05.654421 1942 tablet_bootstrap.cc:492] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: Bootstrap starting.
I20260809 08:42:05.654902 1942 tablet_bootstrap.cc:654] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:05.655112 1943 tablet_bootstrap.cc:492] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682: Bootstrap starting.
I20260809 08:42:05.655537 1943 tablet_bootstrap.cc:654] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:05.656199 1943 tablet_bootstrap.cc:492] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682: No bootstrap required, opened a new log
I20260809 08:42:05.656266 1943 ts_tablet_manager.cc:1397] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:05.656329 1980 ts_tablet_manager.cc:1918] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:42:05.656395 1980 log.cc:1196] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/c9490e968baf47e280707b2b6ab6dd51
I20260809 08:42:05.656428 1943 raft_consensus.cc:348] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } }
I20260809 08:42:05.656503 1943 raft_consensus.cc:374] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:05.656528 1943 raft_consensus.cc:729] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1485ee498ee94f95a1ae256b8d276682, State: Initialized, Role: FOLLOWER
I20260809 08:42:05.656574 1943 consensus_queue.cc:260] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } }
I20260809 08:42:05.656689 1943 ts_tablet_manager.cc:1428] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:05.656721 1980 ts_tablet_manager.cc:1939] T c9490e968baf47e280707b2b6ab6dd51 P 63ce77d13705406fa5e8973766dd6294: Deleting consensus metadata
I20260809 08:42:05.656843 1949 tablet_bootstrap.cc:492] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5: Bootstrap starting.
I20260809 08:42:05.656998 1979 ts_tablet_manager.cc:1905] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:05.657115 1177 catalog_manager.cc:4841] TS 63ce77d13705406fa5e8973766dd6294 (127.31.63.67:46607): tablet c9490e968baf47e280707b2b6ab6dd51 (table test-table [id=addb9879574846ab998e7fe52348ba51]) successfully deleted
I20260809 08:42:05.657287 1949 tablet_bootstrap.cc:654] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:05.657689 1942 tablet_bootstrap.cc:492] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: No bootstrap required, opened a new log
I20260809 08:42:05.657733 1942 ts_tablet_manager.cc:1397] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260809 08:42:05.657881 1942 raft_consensus.cc:348] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } }
I20260809 08:42:05.657945 1942 raft_consensus.cc:374] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:05.657966 1942 raft_consensus.cc:729] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63ce77d13705406fa5e8973766dd6294, State: Initialized, Role: FOLLOWER
I20260809 08:42:05.658010 1942 consensus_queue.cc:260] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } }
I20260809 08:42:05.658085 1942 ts_tablet_manager.cc:1428] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:05.658890 1949 tablet_bootstrap.cc:492] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5: No bootstrap required, opened a new log
I20260809 08:42:05.658948 1949 ts_tablet_manager.cc:1397] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 08:42:05.659036 1979 ts_tablet_manager.cc:1918] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:42:05.659102 1979 log.cc:1196] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/c9490e968baf47e280707b2b6ab6dd51
I20260809 08:42:05.659117 1949 raft_consensus.cc:348] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } }
I20260809 08:42:05.659209 1949 raft_consensus.cc:374] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:05.659245 1949 raft_consensus.cc:729] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee53ae8640c4b788191e4e7ca4451e5, State: Initialized, Role: FOLLOWER
I20260809 08:42:05.659291 1949 consensus_queue.cc:260] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } }
I20260809 08:42:05.659392 1949 ts_tablet_manager.cc:1428] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:05.659814 1979 ts_tablet_manager.cc:1939] T c9490e968baf47e280707b2b6ab6dd51 P 1485ee498ee94f95a1ae256b8d276682: Deleting consensus metadata
I20260809 08:42:05.660259 1981 tablet_replica.cc:330] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: stopping tablet replica
I20260809 08:42:05.660287 1176 catalog_manager.cc:4841] TS 1485ee498ee94f95a1ae256b8d276682 (127.31.63.65:34369): tablet c9490e968baf47e280707b2b6ab6dd51 (table test-table [id=addb9879574846ab998e7fe52348ba51]) successfully deleted
I20260809 08:42:05.660374 1981 raft_consensus.cc:2229] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:05.660496 1981 raft_consensus.cc:2258] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:05.661042 1981 ts_tablet_manager.cc:1905] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:05.662693 1981 ts_tablet_manager.cc:1918] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:42:05.662772 1981 log.cc:1196] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/c9490e968baf47e280707b2b6ab6dd51
I20260809 08:42:05.663021 1981 ts_tablet_manager.cc:1939] T c9490e968baf47e280707b2b6ab6dd51 P 4ee53ae8640c4b788191e4e7ca4451e5: Deleting consensus metadata
I20260809 08:42:05.663434 1177 catalog_manager.cc:4841] TS 4ee53ae8640c4b788191e4e7ca4451e5 (127.31.63.66:33663): tablet c9490e968baf47e280707b2b6ab6dd51 (table test-table [id=addb9879574846ab998e7fe52348ba51]) successfully deleted
W20260809 08:42:05.723942 1912 tablet.cc:2369] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:42:05.782459 1970 raft_consensus.cc:482] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:05.782567 1970 raft_consensus.cc:504] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } }
I20260809 08:42:05.782783 1970 leader_election.cc:288] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1485ee498ee94f95a1ae256b8d276682 (127.31.63.65:34369), 4ee53ae8640c4b788191e4e7ca4451e5 (127.31.63.66:33663)
I20260809 08:42:05.783039 1644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e788790f47147588ea1aba54dedfc8f" candidate_uuid: "63ce77d13705406fa5e8973766dd6294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" is_pre_election: true
I20260809 08:42:05.783073 1446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e788790f47147588ea1aba54dedfc8f" candidate_uuid: "63ce77d13705406fa5e8973766dd6294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1485ee498ee94f95a1ae256b8d276682" is_pre_election: true
I20260809 08:42:05.783171 1644 raft_consensus.cc:2454] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63ce77d13705406fa5e8973766dd6294 in term 0.
I20260809 08:42:05.783165 1446 raft_consensus.cc:2454] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63ce77d13705406fa5e8973766dd6294 in term 0.
I20260809 08:42:05.783383 1747 leader_election.cc:302] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1485ee498ee94f95a1ae256b8d276682, 63ce77d13705406fa5e8973766dd6294; no voters:
I20260809 08:42:05.783511 1970 raft_consensus.cc:2781] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:05.783584 1970 raft_consensus.cc:482] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:05.783618 1970 raft_consensus.cc:3037] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:05.784412 1970 raft_consensus.cc:504] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } }
I20260809 08:42:05.784569 1970 leader_election.cc:288] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [CANDIDATE]: Term 1 election: Requested vote from peers 1485ee498ee94f95a1ae256b8d276682 (127.31.63.65:34369), 4ee53ae8640c4b788191e4e7ca4451e5 (127.31.63.66:33663)
I20260809 08:42:05.784740 1446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e788790f47147588ea1aba54dedfc8f" candidate_uuid: "63ce77d13705406fa5e8973766dd6294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1485ee498ee94f95a1ae256b8d276682"
I20260809 08:42:05.784752 1644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e788790f47147588ea1aba54dedfc8f" candidate_uuid: "63ce77d13705406fa5e8973766dd6294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee53ae8640c4b788191e4e7ca4451e5"
I20260809 08:42:05.784821 1446 raft_consensus.cc:3037] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:05.784842 1644 raft_consensus.cc:3037] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:05.785454 1644 raft_consensus.cc:2454] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63ce77d13705406fa5e8973766dd6294 in term 1.
I20260809 08:42:05.785611 1446 raft_consensus.cc:2454] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63ce77d13705406fa5e8973766dd6294 in term 1.
I20260809 08:42:05.785686 1748 leader_election.cc:302] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee53ae8640c4b788191e4e7ca4451e5, 63ce77d13705406fa5e8973766dd6294; no voters:
I20260809 08:42:05.785775 1970 raft_consensus.cc:2781] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:05.785845 1970 raft_consensus.cc:686] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 1 LEADER]: Becoming Leader. State: Replica: 63ce77d13705406fa5e8973766dd6294, State: Running, Role: LEADER
I20260809 08:42:05.785899 1970 consensus_queue.cc:237] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } }
I20260809 08:42:05.786554 1200 catalog_manager.cc:5495] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 63ce77d13705406fa5e8973766dd6294 (127.31.63.67). New cstate: current_term: 1 leader_uuid: "63ce77d13705406fa5e8973766dd6294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } health_report { overall_health: HEALTHY } } }
Aug 09 08:42:05 dist-test-slave-gkw7 krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264924, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:05.829007 1999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.822244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:44469 (local address 127.0.0.1:46498)
0809 08:42:05.822402 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:05.822413 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:05.822481 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:05.822604 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:05.822605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:05.822920 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:05.823005 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.823008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.823638 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.823639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.824229 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:05.824231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.824557 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.824558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:05.824600 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.825094 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:05.825099 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:05.826473 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:05.828022 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.828029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.828031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.828214 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.828393 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:05.828394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:05.828394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:05.828449 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:05.828679 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:05.828687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:05.828744 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 08:42:05.828816 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:42:05.828910 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260809 08:42:05.829054 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:05.822330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46498 (local address 127.31.63.126:44469)
0809 08:42:05.822415 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:05.822417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:05.822483 (+ 66us) server_negotiation.cc:407] Connection header received
0809 08:42:05.822525 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:05.822526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:05.822535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:05.822588 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:05.823081 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.823573 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.824297 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:05.824493 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:05.824543 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:05.826552 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:05.826553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:05.826554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:05.826560 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:05.827956 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.828285 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.828286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.828286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.828337 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:05.828501 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:05.828501 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:05.828501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:05.828617 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:05.828667 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:05.828851 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 08:42:05.828957 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":52}
I20260809 08:42:05.831250 1200 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:46498:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 08:42:05.831553 1200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:05.833238 1614 tablet_service.cc:1467] Processing CreateTablet for tablet 4ffbd3805f1c4840af5bc1dc6b9210d1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17b0d62e8c7c47eea98e5ea945e1c545]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:42:05.833511 1614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ffbd3805f1c4840af5bc1dc6b9210d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:05.834923 1949 tablet_bootstrap.cc:492] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: Bootstrap starting.
I20260809 08:42:05.833246 1815 tablet_service.cc:1467] Processing CreateTablet for tablet 4ffbd3805f1c4840af5bc1dc6b9210d1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17b0d62e8c7c47eea98e5ea945e1c545]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:42:05.835358 1949 tablet_bootstrap.cc:654] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:05.835386 1815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ffbd3805f1c4840af5bc1dc6b9210d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:05.833246 1417 tablet_service.cc:1467] Processing CreateTablet for tablet 4ffbd3805f1c4840af5bc1dc6b9210d1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17b0d62e8c7c47eea98e5ea945e1c545]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:42:05.835930 1417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ffbd3805f1c4840af5bc1dc6b9210d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:05.835995 1949 tablet_bootstrap.cc:492] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: No bootstrap required, opened a new log
I20260809 08:42:05.836030 1949 ts_tablet_manager.cc:1397] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:05.836150 1949 raft_consensus.cc:348] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.836210 1949 raft_consensus.cc:374] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:05.836223 1949 raft_consensus.cc:729] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee53ae8640c4b788191e4e7ca4451e5, State: Initialized, Role: FOLLOWER
I20260809 08:42:05.836284 1949 consensus_queue.cc:260] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.836392 1949 ts_tablet_manager.cc:1428] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:05.836445 1955 tablet_replica.cc:439] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } }
I20260809 08:42:05.836522 1955 tablet_replica.cc:442] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:05.836980 1942 tablet_bootstrap.cc:492] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: Bootstrap starting.
I20260809 08:42:05.837321 1942 tablet_bootstrap.cc:654] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:05.837512 1943 tablet_bootstrap.cc:492] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: Bootstrap starting.
I20260809 08:42:05.838014 1943 tablet_bootstrap.cc:654] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:05.838322 1942 tablet_bootstrap.cc:492] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: No bootstrap required, opened a new log
I20260809 08:42:05.838375 1942 ts_tablet_manager.cc:1397] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:05.838490 1942 raft_consensus.cc:348] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.838541 1942 raft_consensus.cc:374] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:05.838553 1942 raft_consensus.cc:729] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63ce77d13705406fa5e8973766dd6294, State: Initialized, Role: FOLLOWER
I20260809 08:42:05.838609 1942 consensus_queue.cc:260] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.838644 1943 tablet_bootstrap.cc:492] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: No bootstrap required, opened a new log
I20260809 08:42:05.838678 1943 ts_tablet_manager.cc:1397] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:05.838701 1942 ts_tablet_manager.cc:1428] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:05.838729 1970 tablet_replica.cc:439] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } }
I20260809 08:42:05.838815 1970 tablet_replica.cc:442] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:05.838857 1943 raft_consensus.cc:348] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.838940 1943 raft_consensus.cc:374] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:05.838972 1943 raft_consensus.cc:729] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1485ee498ee94f95a1ae256b8d276682, State: Initialized, Role: FOLLOWER
I20260809 08:42:05.839035 1943 consensus_queue.cc:260] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:05.839136 1943 ts_tablet_manager.cc:1428] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:42:05.839282 1953 tablet_replica.cc:439] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } }
I20260809 08:42:05.839380 1953 tablet_replica.cc:442] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:06.148185 1955 raft_consensus.cc:482] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:06.148310 1955 raft_consensus.cc:504] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:06.148681 1955 leader_election.cc:288] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1485ee498ee94f95a1ae256b8d276682 (127.31.63.65:34369), 63ce77d13705406fa5e8973766dd6294 (127.31.63.67:46607)
I20260809 08:42:06.151902 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.148904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:34369 (local address 127.31.63.66:53223)
0809 08:42:06.149115 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:06.149128 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:06.149240 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:06.149434 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:06.149435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:06.149437 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:06.149514 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:06.149516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.150195 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.150197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:06.151050 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:06.151053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.151602 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.151603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:06.151684 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.151689 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:42:06.151749 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:42:06.151782 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":78,"thread_start_us":51,"threads_started":1}
I20260809 08:42:06.151901 2007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.148979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:53223 (local address 127.31.63.65:34369)
0809 08:42:06.149092 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:06.149095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:06.149219 (+ 124us) server_negotiation.cc:407] Connection header received
0809 08:42:06.149309 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:06.149310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:06.149338 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:06.149420 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:06.149609 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.150098 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.151158 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.151523 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.151580 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.151588 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:06.151771 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 08:42:06.151822 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
I20260809 08:42:06.152010 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.148904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:46607 (local address 127.31.63.66:57479)
0809 08:42:06.149084 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:06.149098 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:06.149165 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:06.149361 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:06.149362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:06.149365 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:06.149459 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:06.149462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.150209 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.150210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:06.151049 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:06.151051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.151680 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.151681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:06.151756 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.151762 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:42:06.151821 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:42:06.151858 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":25,"thread_start_us":77,"threads_started":1}
I20260809 08:42:06.152135 1446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffbd3805f1c4840af5bc1dc6b9210d1" candidate_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1485ee498ee94f95a1ae256b8d276682" is_pre_election: true
I20260809 08:42:06.152251 1446 raft_consensus.cc:2454] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ee53ae8640c4b788191e4e7ca4451e5 in term 0.
I20260809 08:42:06.152347 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.149036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:57479 (local address 127.31.63.67:46607)
0809 08:42:06.149133 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:06.149135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:06.149151 (+ 16us) server_negotiation.cc:407] Connection header received
0809 08:42:06.149258 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:06.149259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:06.149272 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:06.149347 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:06.149572 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.150100 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.151155 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.151573 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.151646 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.151656 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:06.151840 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 08:42:06.151893 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":47,"threads_started":1}
I20260809 08:42:06.152452 1546 leader_election.cc:302] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1485ee498ee94f95a1ae256b8d276682, 4ee53ae8640c4b788191e4e7ca4451e5; no voters:
I20260809 08:42:06.152578 1955 raft_consensus.cc:2781] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:06.152637 1955 raft_consensus.cc:482] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:06.152652 1955 raft_consensus.cc:3037] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:06.153357 1955 raft_consensus.cc:504] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:06.153509 1955 leader_election.cc:288] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [CANDIDATE]: Term 1 election: Requested vote from peers 1485ee498ee94f95a1ae256b8d276682 (127.31.63.65:34369), 63ce77d13705406fa5e8973766dd6294 (127.31.63.67:46607)
I20260809 08:42:06.153611 1834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffbd3805f1c4840af5bc1dc6b9210d1" candidate_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63ce77d13705406fa5e8973766dd6294" is_pre_election: true
I20260809 08:42:06.153663 1446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffbd3805f1c4840af5bc1dc6b9210d1" candidate_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1485ee498ee94f95a1ae256b8d276682"
I20260809 08:42:06.153694 1833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ffbd3805f1c4840af5bc1dc6b9210d1" candidate_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63ce77d13705406fa5e8973766dd6294"
I20260809 08:42:06.153761 1446 raft_consensus.cc:3037] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:06.153788 1834 raft_consensus.cc:2454] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ee53ae8640c4b788191e4e7ca4451e5 in term 0.
I20260809 08:42:06.154386 1446 raft_consensus.cc:2454] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ee53ae8640c4b788191e4e7ca4451e5 in term 1.
I20260809 08:42:06.154562 1546 leader_election.cc:302] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1485ee498ee94f95a1ae256b8d276682, 4ee53ae8640c4b788191e4e7ca4451e5; no voters: 63ce77d13705406fa5e8973766dd6294
I20260809 08:42:06.154663 1955 raft_consensus.cc:2781] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:06.154796 1955 raft_consensus.cc:686] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 LEADER]: Becoming Leader. State: Replica: 4ee53ae8640c4b788191e4e7ca4451e5, State: Running, Role: LEADER
I20260809 08:42:06.154868 1955 consensus_queue.cc:237] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } }
I20260809 08:42:06.155189 2011 tablet_replica.cc:439] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: TxnStatusTablet state changed. Reason: New leader 4ee53ae8640c4b788191e4e7ca4451e5. Latest consensus state: current_term: 1 leader_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } }
I20260809 08:42:06.155278 2011 tablet_replica.cc:442] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:42:06.155457 2016 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 08:42:06.155567 2016 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 08:42:06.155660 1200 catalog_manager.cc:5495] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ee53ae8640c4b788191e4e7ca4451e5 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } health_report { overall_health: HEALTHY } } }
I20260809 08:42:06.192226 1999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.189404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:33663 (local address 127.0.0.1:45022)
0809 08:42:06.189478 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:06.189490 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:06.189583 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:06.189814 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:06.189815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:06.189817 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:42:06.189874 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:06.189876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.190587 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.190588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:06.191225 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:06.191227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.191643 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.191644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:06.191717 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.191739 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:42:06.192027 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:42:06.192038 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 08:42:06.192088 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:42:06.192142 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 08:42:06.192239 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.189471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.31.63.66:33663)
0809 08:42:06.189600 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:06.189602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:06.189612 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:06.189683 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:06.189684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:06.189706 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:06.189806 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:06.189972 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.190495 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.191323 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.191553 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.191630 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.191842 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:06.191957 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:06.192043 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:06.192092 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 08:42:06.192115 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":89,"threads_started":1}
I20260809 08:42:06.193600 1446 raft_consensus.cc:1261] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682 [term 1 FOLLOWER]: Refusing update from remote peer 4ee53ae8640c4b788191e4e7ca4451e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:06.193614 1833 raft_consensus.cc:3037] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:06.193970 2011 consensus_queue.cc:1035] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:06.194716 1833 raft_consensus.cc:1261] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294 [term 1 FOLLOWER]: Refusing update from remote peer 4ee53ae8640c4b788191e4e7ca4451e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:06.194775 1953 tablet_replica.cc:439] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: TxnStatusTablet state changed. Reason: New leader 4ee53ae8640c4b788191e4e7ca4451e5. Latest consensus state: current_term: 1 leader_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } }
I20260809 08:42:06.194872 1953 tablet_replica.cc:442] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:06.194988 2011 consensus_queue.cc:1035] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:06.195683 1970 tablet_replica.cc:439] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: TxnStatusTablet state changed. Reason: New leader 4ee53ae8640c4b788191e4e7ca4451e5. Latest consensus state: current_term: 1 leader_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } }
I20260809 08:42:06.195777 1970 tablet_replica.cc:442] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:06.195966 2011 tablet_replica.cc:439] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } }
I20260809 08:42:06.196067 2011 tablet_replica.cc:442] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:42:06.196781 1953 tablet_replica.cc:439] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } }
I20260809 08:42:06.196869 1953 tablet_replica.cc:442] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 1485ee498ee94f95a1ae256b8d276682: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:06.197010 1955 tablet_replica.cc:439] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } }
I20260809 08:42:06.197114 1955 tablet_replica.cc:442] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 4ee53ae8640c4b788191e4e7ca4451e5: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:42:06.197502 2023 mvcc.cc:205] Tried to move back new op lower bound from 7316541137686556672 to 7316541137530785792. Current Snapshot: MvccSnapshot[applied={T|T < 7316541137686556672 or (T in {7316541137686556672})}]
I20260809 08:42:06.197772 1970 tablet_replica.cc:439] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 } } peers { permanent_uuid: "63ce77d13705406fa5e8973766dd6294" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 46607 } } peers { permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 } } }
I20260809 08:42:06.197851 1970 tablet_replica.cc:442] T 4ffbd3805f1c4840af5bc1dc6b9210d1 P 63ce77d13705406fa5e8973766dd6294: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:06.203943 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.200889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:54183 (local address 127.31.63.66:33663)
0809 08:42:06.200935 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:06.200938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:06.201064 (+ 126us) server_negotiation.cc:407] Connection header received
0809 08:42:06.201160 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:06.201161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:06.201173 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:06.201236 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:06.201371 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.202083 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.203185 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.203370 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.203425 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.203622 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:06.203660 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:06.203713 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:06.203816 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 08:42:06.203843 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:42:06.204013 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.200828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:33663 (local address 127.31.63.67:54183)
0809 08:42:06.200985 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:06.200997 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:06.201063 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:06.201249 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:06.201250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:06.201252 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:42:06.201308 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:06.201310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.202161 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.202162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:06.203100 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:06.203103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.203444 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.203445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:06.203508 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.203528 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:42:06.203754 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:42:06.203756 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:42:06.203799 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:42:06.203881 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":85,"threads_started":1}
I20260809 08:42:06.206779 1446 raft_consensus.cc:1261] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682 [term 1 FOLLOWER]: Refusing update from remote peer 63ce77d13705406fa5e8973766dd6294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:06.206776 1644 raft_consensus.cc:1261] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 FOLLOWER]: Refusing update from remote peer 63ce77d13705406fa5e8973766dd6294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:06.207017 1970 consensus_queue.cc:1035] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee53ae8640c4b788191e4e7ca4451e5" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:06.207115 1959 consensus_queue.cc:1035] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1485ee498ee94f95a1ae256b8d276682" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:06.216393 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.213492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:46607 (local address 127.31.63.66:55495)
0809 08:42:06.213662 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:06.213674 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:06.213738 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:06.213867 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:06.213867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:06.213868 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:42:06.213915 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:06.213916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.214791 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.214793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:06.215541 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:06.215545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.216006 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.216007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:06.216047 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.216063 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:42:06.216253 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:42:06.216254 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:42:06.216289 (+ 35us) client_negotiation.cc:240] Negotiation successful
0809 08:42:06.216324 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260809 08:42:06.216387 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.213597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:55495 (local address 127.31.63.67:46607)
0809 08:42:06.213640 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:06.213642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:06.213744 (+ 102us) server_negotiation.cc:407] Connection header received
0809 08:42:06.213794 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:06.213794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:06.213807 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:06.213866 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:06.213960 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.214712 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.215629 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.215927 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.216003 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.216140 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:06.216177 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:06.216238 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:06.216292 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 08:42:06.216314 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:42:06.224334 1200 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:06.224458 1200 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:06.225111 1200 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7be99251fc05410cb678240287118409: Sending DeleteTablet for 3 replicas of tablet 0e788790f47147588ea1aba54dedfc8f
I20260809 08:42:06.225374 1417 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e788790f47147588ea1aba54dedfc8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:06 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:46348
I20260809 08:42:06.225407 1815 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e788790f47147588ea1aba54dedfc8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:06 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:34788
I20260809 08:42:06.225423 1614 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e788790f47147588ea1aba54dedfc8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:06 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:43406
I20260809 08:42:06.225657 2051 tablet_replica.cc:330] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: stopping tablet replica
I20260809 08:42:06.225726 2051 raft_consensus.cc:2229] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:42:06.225798 2051 raft_consensus.cc:2258] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:06.226483 2044 tablet_replica.cc:330] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682: stopping tablet replica
I20260809 08:42:06.226537 2051 ts_tablet_manager.cc:1905] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:06.226557 2044 raft_consensus.cc:2229] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:06.226594 2044 raft_consensus.cc:2258] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:06.226845 2044 ts_tablet_manager.cc:1905] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:06.228427 2051 ts_tablet_manager.cc:1918] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:42:06.228487 2051 log.cc:1196] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/0e788790f47147588ea1aba54dedfc8f
I20260809 08:42:06.228590 2044 ts_tablet_manager.cc:1918] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:42:06.228643 2044 log.cc:1196] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/0e788790f47147588ea1aba54dedfc8f
I20260809 08:42:06.228703 2051 ts_tablet_manager.cc:1939] T 0e788790f47147588ea1aba54dedfc8f P 63ce77d13705406fa5e8973766dd6294: Deleting consensus metadata
I20260809 08:42:06.228901 2044 ts_tablet_manager.cc:1939] T 0e788790f47147588ea1aba54dedfc8f P 1485ee498ee94f95a1ae256b8d276682: Deleting consensus metadata
I20260809 08:42:06.229113 1177 catalog_manager.cc:4841] TS 63ce77d13705406fa5e8973766dd6294 (127.31.63.67:46607): tablet 0e788790f47147588ea1aba54dedfc8f (table test-table [id=cef5cd23cf654e3384a9ffb3a87759a7]) successfully deleted
I20260809 08:42:06.229453 2047 tablet_replica.cc:330] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5: stopping tablet replica
I20260809 08:42:06.229519 2047 raft_consensus.cc:2229] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:06.229506 1176 catalog_manager.cc:4841] TS 1485ee498ee94f95a1ae256b8d276682 (127.31.63.65:34369): tablet 0e788790f47147588ea1aba54dedfc8f (table test-table [id=cef5cd23cf654e3384a9ffb3a87759a7]) successfully deleted
I20260809 08:42:06.229583 2047 raft_consensus.cc:2258] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:06.229853 2047 ts_tablet_manager.cc:1905] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:06.231515 2047 ts_tablet_manager.cc:1918] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:42:06.231585 2047 log.cc:1196] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/0e788790f47147588ea1aba54dedfc8f
I20260809 08:42:06.231810 2047 ts_tablet_manager.cc:1939] T 0e788790f47147588ea1aba54dedfc8f P 4ee53ae8640c4b788191e4e7ca4451e5: Deleting consensus metadata
I20260809 08:42:06.232126 1177 catalog_manager.cc:4841] TS 4ee53ae8640c4b788191e4e7ca4451e5 (127.31.63.66:33663): tablet 0e788790f47147588ea1aba54dedfc8f (table test-table [id=cef5cd23cf654e3384a9ffb3a87759a7]) successfully deleted
Aug 09 08:42:06 dist-test-slave-gkw7 krb5kdc[1124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264925, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:42:06.235903 2007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.226563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.31.63.65:34369)
0809 08:42:06.226613 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:06.226616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:06.226903 (+ 287us) server_negotiation.cc:407] Connection header received
0809 08:42:06.227050 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:06.227051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:06.227063 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:06.227092 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:06.227888 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.228666 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.229972 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.230171 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.230251 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.232819 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:06.232830 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:06.232830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:06.232837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:06.234637 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:06.235026 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:06.235030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:06.235030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:06.235117 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:06.235329 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:06.235330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:06.235330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:06.235485 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:06.235643 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:06.235697 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 08:42:06.235795 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 08:42:06.236268 1503 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51254
I20260809 08:42:06.243674 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:06.237361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58702 (local address 127.31.63.126:44469)
0809 08:42:06.237419 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:06.237421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:06.237431 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:06.237503 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:06.237504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:06.237514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:06.237608 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:06.238095 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.238606 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.239313 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:06.239527 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:06.239619 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:06.241005 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:06.241006 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:06.241007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:06.241014 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:06.242515 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:06.242887 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:06.242888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:06.242889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:06.242951 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:06.243159 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:06.243159 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:06.243160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:06.243281 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:06.243384 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:06.243493 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 08:42:06.243568 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 08:42:06.243940 1200 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58702
I20260809 08:42:06.244500 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 1319
I20260809 08:42:06.262308 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 1525
I20260809 08:42:06.275764 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 1723
I20260809 08:42:06.291622 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 1149
2026-08-09T08:42:06Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2939 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:42:06 dist-test-slave-gkw7 krb5kdc[2060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:42:06 dist-test-slave-gkw7 krb5kdc[2060](info): set up 2 sockets
Aug 09 08:42:06 dist-test-slave-gkw7 krb5kdc[2060](info): commencing operation
krb5kdc: starting...
W20260809 08:42:07.569772 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:42:07 dist-test-slave-gkw7 krb5kdc[2060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:42:07.698513 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33763
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:07.780596 2080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:07.780812 2080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:07.780835 2080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:07.781948 2080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:07.781984 2080 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 08:42:07.781997 2080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:07.782011 2080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:07.782022 2080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:07.782034 2080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:07.783473 2080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33763
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:07.783674 2080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:07.783855 2080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:07.788921 2089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:07.788955 2094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:07.788940 2088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:07.789250 2080 server_base.cc:1034] running on GCE node
I20260809 08:42:07.789565 2080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:07.789855 2080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:07.791018 2080 hybrid_clock.cc:648] HybridClock initialized: now 1786264927791013 us; error 41 us; skew 500 ppm
Aug 09 08:42:07 dist-test-slave-gkw7 krb5kdc[2060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264927, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:07.793499 2080 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:42:07.793987 2080 webserver.cc:460] Webserver started at http://127.31.63.126:32819/ using document root <none> and password file <none>
I20260809 08:42:07.794184 2080 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:07.794234 2080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:07.794324 2080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:07.795307 2080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "98bb4e14d7664a9d8bb774376fdd06e5"
format_stamp: "Formatted at 2026-08-09 08:42:07 on dist-test-slave-gkw7"
server_key: "8b673477d8237fa1903e801427a054a1"
server_key_iv: "d9f8420b70af9a6eda217655e9ece5a6"
server_key_version: "encryptionkey@0"
I20260809 08:42:07.795603 2080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "98bb4e14d7664a9d8bb774376fdd06e5"
format_stamp: "Formatted at 2026-08-09 08:42:07 on dist-test-slave-gkw7"
server_key: "8b673477d8237fa1903e801427a054a1"
server_key_iv: "d9f8420b70af9a6eda217655e9ece5a6"
server_key_version: "encryptionkey@0"
I20260809 08:42:07.796871 2080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:07.797585 2102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:07.797792 2080 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:07.797889 2080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "98bb4e14d7664a9d8bb774376fdd06e5"
format_stamp: "Formatted at 2026-08-09 08:42:07 on dist-test-slave-gkw7"
server_key: "8b673477d8237fa1903e801427a054a1"
server_key_iv: "d9f8420b70af9a6eda217655e9ece5a6"
server_key_version: "encryptionkey@0"
I20260809 08:42:07.797946 2080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:07.813360 2080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:07.815734 2080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:07.815865 2080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:07.820681 2080 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:33763
I20260809 08:42:07.820755 2190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:33763 every 8 connection(s)
I20260809 08:42:07.821118 2080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:07.821887 2192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:07.823962 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 2080
I20260809 08:42:07.824062 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:07.824265 31997 external_mini_cluster.cc:1428] Setting key a14d1e5df209558bba14aa3e0d8a7e8b
I20260809 08:42:07.824505 2192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5: Bootstrap starting.
I20260809 08:42:07.825225 2192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:07.825572 2192 log.cc:824] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:07.826277 2192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5: No bootstrap required, opened a new log
I20260809 08:42:07.828112 2192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bb4e14d7664a9d8bb774376fdd06e5" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33763 } }
I20260809 08:42:07.828281 2192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:07.828346 2192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98bb4e14d7664a9d8bb774376fdd06e5, State: Initialized, Role: FOLLOWER
I20260809 08:42:07.828521 2192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bb4e14d7664a9d8bb774376fdd06e5" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33763 } }
I20260809 08:42:07.828617 2192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:07.828662 2192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:07.828717 2192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:07.829244 2192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bb4e14d7664a9d8bb774376fdd06e5" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33763 } }
I20260809 08:42:07.829365 2192 leader_election.cc:302] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98bb4e14d7664a9d8bb774376fdd06e5; no voters:
I20260809 08:42:07.829574 2192 leader_election.cc:288] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:07.829598 2202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:07.829782 2202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [term 1 LEADER]: Becoming Leader. State: Replica: 98bb4e14d7664a9d8bb774376fdd06e5, State: Running, Role: LEADER
I20260809 08:42:07.829861 2192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:07.829908 2202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bb4e14d7664a9d8bb774376fdd06e5" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33763 } }
I20260809 08:42:07.830333 2202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98bb4e14d7664a9d8bb774376fdd06e5. Latest consensus state: current_term: 1 leader_uuid: "98bb4e14d7664a9d8bb774376fdd06e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bb4e14d7664a9d8bb774376fdd06e5" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33763 } } }
I20260809 08:42:07.830585 2202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:07.830510 2203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98bb4e14d7664a9d8bb774376fdd06e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98bb4e14d7664a9d8bb774376fdd06e5" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33763 } } }
I20260809 08:42:07.831004 2203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5 [sys.catalog]: This master's current role is: LEADER
Aug 09 08:42:07 dist-test-slave-gkw7 krb5kdc[2060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
W20260809 08:42:07.831744 2225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:42:07.831802 2225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:42:07.831854 2215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:07.832041 2215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:07.833684 2215 catalog_manager.cc:1347] Generated new cluster ID: aaf896d26c3f4123ab55176146cc8518
I20260809 08:42:07.833747 2215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:07.835464 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:07.824798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41464 (local address 127.31.63.126:33763)
0809 08:42:07.825009 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:07.825019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:07.825055 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:42:07.825637 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:07.825645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:07.825724 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:07.825883 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:07.826582 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:07.827477 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:07.828269 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:07.828861 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:07.828982 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:07.831955 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:07.831959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:07.831959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:07.831971 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:07.833968 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:07.834320 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:07.834323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:07.834323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:07.834428 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:07.834584 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:07.834584 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:07.834585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:07.834856 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:07.834978 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:07.835131 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 08:42:07.835267 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":107,"threads_started":1}
I20260809 08:42:07.846289 2215 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:07.846833 2215 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:07.854705 2215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5: Generated new TSK 0
I20260809 08:42:07.854904 2215 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:42:07.888348 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33763
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:07.972784 2232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:07.973001 2232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:07.973026 2232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:07.974229 2232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:07.974278 2232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:07.974299 2232 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 08:42:07.974311 2232 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:42:07.974328 2232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:07.975831 2232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.63.126:33763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:07.976089 2232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:07.976358 2232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:07.981683 2246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:07.981690 2247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:07.981694 2249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:07.981839 2232 server_base.cc:1034] running on GCE node
I20260809 08:42:07.982122 2232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:07.982390 2232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:07.983580 2232 hybrid_clock.cc:648] HybridClock initialized: now 1786264927983567 us; error 26 us; skew 500 ppm
Aug 09 08:42:07 dist-test-slave-gkw7 krb5kdc[2060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264927, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:07.986114 2232 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:42:07.986580 2232 webserver.cc:460] Webserver started at http://127.31.63.65:39471/ using document root <none> and password file <none>
I20260809 08:42:07.986855 2232 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:07.986929 2232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:07.987082 2232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:07.988157 2232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "e79bd558378c443cb7139a70215a0c33"
format_stamp: "Formatted at 2026-08-09 08:42:07 on dist-test-slave-gkw7"
server_key: "a4a15862fd882b1d01b41c5b6cf5d516"
server_key_iv: "218b793683daa79823794f74475aa62b"
server_key_version: "encryptionkey@0"
I20260809 08:42:07.988552 2232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "e79bd558378c443cb7139a70215a0c33"
format_stamp: "Formatted at 2026-08-09 08:42:07 on dist-test-slave-gkw7"
server_key: "a4a15862fd882b1d01b41c5b6cf5d516"
server_key_iv: "218b793683daa79823794f74475aa62b"
server_key_version: "encryptionkey@0"
I20260809 08:42:07.990000 2232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:07.990728 2259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:07.990904 2232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:07.990998 2232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "e79bd558378c443cb7139a70215a0c33"
format_stamp: "Formatted at 2026-08-09 08:42:07 on dist-test-slave-gkw7"
server_key: "a4a15862fd882b1d01b41c5b6cf5d516"
server_key_iv: "218b793683daa79823794f74475aa62b"
server_key_version: "encryptionkey@0"
I20260809 08:42:07.991081 2232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:08.002420 2232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:08.004370 2232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:08.004554 2232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:08.005113 2232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:08.005179 2232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:08.005219 2232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:08.005263 2232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:42:08 dist-test-slave-gkw7 krb5kdc[2060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264927, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:08.012751 2232 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:35985
I20260809 08:42:08.012805 2429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:35985 every 8 connection(s)
I20260809 08:42:08.013191 2232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:08.013868 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 2232
I20260809 08:42:08.013953 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:08.014148 31997 external_mini_cluster.cc:1428] Setting key 8e8b7248d7a201372b9e367146dfff3c
I20260809 08:42:08.016409 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.005706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:58979 (local address 127.31.63.126:33763)
0809 08:42:08.005777 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.005779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.006008 (+ 229us) server_negotiation.cc:407] Connection header received
0809 08:42:08.006564 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.006565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.006580 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.006633 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:08.007370 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.008199 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.009809 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.010161 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.010237 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.013106 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:08.013107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:08.013108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:08.013116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:08.014896 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.015282 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.015284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.015284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.015355 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.015587 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.015588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.015588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.015744 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:08.015820 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.016129 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.016254 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 08:42:08.016507 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.005709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33763 (local address 127.31.63.65:58979)
0809 08:42:08.005882 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.005920 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.006451 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.006726 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.006736 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.007059 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:08.007269 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.007274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.008296 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.008297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.009695 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.009698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.010300 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.010302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.010480 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.010982 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:08.010993 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:08.013007 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:08.014974 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.014981 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.014982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.015205 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.015424 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.015425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.015425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.015524 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.015823 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:08.015828 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:08.015983 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.016086 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.016253 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260809 08:42:08.017362 2430 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33763
I20260809 08:42:08.017477 2430 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:08.017735 2430 heartbeater.cc:511] Master 127.31.63.126:33763 requested a full tablet report, sending...
I20260809 08:42:08.018354 2128 ts_manager.cc:194] Registered new tserver with Master: e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985)
I20260809 08:42:08.019156 2128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:58979
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
I20260809 08:42:08.024808 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.018538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:53191 (local address 127.31.63.126:33763)
0809 08:42:08.018616 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.018618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.018627 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:42:08.018647 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.018648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.018657 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.018719 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:08.019123 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.019674 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.020810 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.021049 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.021110 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.022285 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:08.022287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:08.022287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:08.022294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:08.023674 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.024086 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.024087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.024087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.024137 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.024334 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.024335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.024335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.024436 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:08.024486 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.024674 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.024744 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 08:42:08.024891 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.018342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33763 (local address 127.31.63.65:53191)
0809 08:42:08.018501 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.018514 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.018571 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.018734 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.018735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.018960 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:08.019036 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.019038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.019780 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.019781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.020717 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.020720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.021135 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.021136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.021221 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.021730 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:08.021737 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:08.022200 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:08.023764 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.023765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.023765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.024007 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.024203 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.024204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.024205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.024275 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.024506 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:08.024507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:08.024565 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.024652 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.024777 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:42:08.045789 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33763
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:08.132421 2439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:08.132656 2439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:08.132699 2439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:08.133930 2439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:08.134001 2439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:08.134032 2439 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 08:42:08.134061 2439 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:42:08.134104 2439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:08.135672 2439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.63.126:33763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:08.135974 2439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:08.136247 2439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:08.141575 2449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:08.141571 2453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:08.142023 2439 server_base.cc:1034] running on GCE node
W20260809 08:42:08.141575 2451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:08.142287 2439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:08.142496 2439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:08.143640 2439 hybrid_clock.cc:648] HybridClock initialized: now 1786264928143626 us; error 40 us; skew 500 ppm
Aug 09 08:42:08 dist-test-slave-gkw7 krb5kdc[2060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264928, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:08.146260 2439 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:42:08.146764 2439 webserver.cc:460] Webserver started at http://127.31.63.66:36391/ using document root <none> and password file <none>
I20260809 08:42:08.147023 2439 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:08.147094 2439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:08.147249 2439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:08.148319 2439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "a7fb0ad9c8a34fb09ee3734ecf600956"
format_stamp: "Formatted at 2026-08-09 08:42:08 on dist-test-slave-gkw7"
server_key: "f6683eb0477e789aabe7be87fecffa50"
server_key_iv: "03e779e7ba78059561bf64a536e16e9a"
server_key_version: "encryptionkey@0"
I20260809 08:42:08.148681 2439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "a7fb0ad9c8a34fb09ee3734ecf600956"
format_stamp: "Formatted at 2026-08-09 08:42:08 on dist-test-slave-gkw7"
server_key: "f6683eb0477e789aabe7be87fecffa50"
server_key_iv: "03e779e7ba78059561bf64a536e16e9a"
server_key_version: "encryptionkey@0"
I20260809 08:42:08.150206 2439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:08.151068 2464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:08.151297 2439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:08.151418 2439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "a7fb0ad9c8a34fb09ee3734ecf600956"
format_stamp: "Formatted at 2026-08-09 08:42:08 on dist-test-slave-gkw7"
server_key: "f6683eb0477e789aabe7be87fecffa50"
server_key_iv: "03e779e7ba78059561bf64a536e16e9a"
server_key_version: "encryptionkey@0"
I20260809 08:42:08.151499 2439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:08.163864 2439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:08.165918 2439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:08.166087 2439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:08.166713 2439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:08.166777 2439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:08.166818 2439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:08.166846 2439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:42:08 dist-test-slave-gkw7 krb5kdc[2060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264928, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:08.174821 2439 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:43571
I20260809 08:42:08.175165 2439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:42:08.175649 2642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:43571 every 8 connection(s)
I20260809 08:42:08.177588 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.167148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:51663 (local address 127.31.63.126:33763)
0809 08:42:08.167236 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.167238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.167508 (+ 270us) server_negotiation.cc:407] Connection header received
0809 08:42:08.168073 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.168075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.168090 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.168166 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:08.168909 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.169792 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.171051 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.171409 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.171485 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.174222 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:08.174223 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:08.174224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:08.174233 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:08.176008 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.176469 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.176470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.176471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.176542 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.176829 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.176830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.176830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.176964 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:08.177050 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.177369 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.177457 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:42:08.177793 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.167141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33763 (local address 127.31.63.66:51663)
0809 08:42:08.167338 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.167376 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.167959 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.168293 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.168303 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.168559 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:08.168813 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.168821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.169916 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.169918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.170951 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.170954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.171521 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.171522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.171660 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.172216 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:08.172228 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:08.174114 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:08.176117 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.176125 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.176127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.176381 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.176618 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.176619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.176619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.176746 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.177039 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:08.177047 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:08.177203 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.177325 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.177506 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260809 08:42:08.178392 2643 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33763
I20260809 08:42:08.178484 2643 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:08.178709 2643 heartbeater.cc:511] Master 127.31.63.126:33763 requested a full tablet report, sending...
I20260809 08:42:08.179177 2127 ts_manager.cc:194] Registered new tserver with Master: a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66:43571)
I20260809 08:42:08.179733 2127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:51663
I20260809 08:42:08.181706 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 2439
I20260809 08:42:08.181846 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:42:08.182066 31997 external_mini_cluster.cc:1428] Setting key dc42149a6d5452b081cd94add4e5d07a
I20260809 08:42:08.185894 2651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.179527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33763 (local address 127.31.63.66:36715)
0809 08:42:08.179698 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.179708 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.179789 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.179933 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.179934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.180207 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:08.180287 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.180289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.181007 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.181008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.181719 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.181722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.182176 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.182178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.182291 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.182815 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:08.182821 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:08.183281 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:08.184743 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.184744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.184745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.184932 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.185149 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.185149 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.185150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.185211 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.185643 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:08.185644 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:08.185685 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.185748 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.185829 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":72,"threads_started":1}
I20260809 08:42:08.186021 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.179587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:36715 (local address 127.31.63.126:33763)
0809 08:42:08.179627 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.179630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.179772 (+ 142us) server_negotiation.cc:407] Connection header received
0809 08:42:08.179853 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.179854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.179866 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.179927 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:08.180456 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.180909 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.181838 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.182075 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.182160 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.183367 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:08.183368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:08.183368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:08.183373 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:08.184655 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.185019 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.185020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.185020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.185081 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.185295 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.185296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.185297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.185561 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:08.185651 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.185797 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.185892 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:42:08.212199 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33763
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:08.296860 2656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:08.297107 2656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:08.297161 2656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:08.298393 2656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:08.298460 2656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:08.298493 2656 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 08:42:08.298521 2656 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:42:08.298561 2656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:08.300029 2656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.63.126:33763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:08.300309 2656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:08.300546 2656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:08.305835 2661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:08.305825 2662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:08.305826 2664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:08.306088 2656 server_base.cc:1034] running on GCE node
I20260809 08:42:08.306325 2656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:08.306569 2656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:08.307711 2656 hybrid_clock.cc:648] HybridClock initialized: now 1786264928307678 us; error 51 us; skew 500 ppm
Aug 09 08:42:08 dist-test-slave-gkw7 krb5kdc[2060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264928, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:08.310420 2656 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:42:08.310859 2656 webserver.cc:460] Webserver started at http://127.31.63.67:34137/ using document root <none> and password file <none>
I20260809 08:42:08.311102 2656 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:08.311173 2656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:08.311337 2656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:08.312404 2656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "dc23d7916aec4c19bd1898e1f1144b13"
format_stamp: "Formatted at 2026-08-09 08:42:08 on dist-test-slave-gkw7"
server_key: "d0aef8441e00343907b7375f06d481a4"
server_key_iv: "74045e0eaa5b749acc62c129547ea011"
server_key_version: "encryptionkey@0"
I20260809 08:42:08.312772 2656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "dc23d7916aec4c19bd1898e1f1144b13"
format_stamp: "Formatted at 2026-08-09 08:42:08 on dist-test-slave-gkw7"
server_key: "d0aef8441e00343907b7375f06d481a4"
server_key_iv: "74045e0eaa5b749acc62c129547ea011"
server_key_version: "encryptionkey@0"
I20260809 08:42:08.314258 2656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:08.315032 2673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:08.315208 2656 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:08.315336 2656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "dc23d7916aec4c19bd1898e1f1144b13"
format_stamp: "Formatted at 2026-08-09 08:42:08 on dist-test-slave-gkw7"
server_key: "d0aef8441e00343907b7375f06d481a4"
server_key_iv: "74045e0eaa5b749acc62c129547ea011"
server_key_version: "encryptionkey@0"
I20260809 08:42:08.315414 2656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:08.322458 2656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:08.324771 2656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:08.324939 2656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:08.325448 2656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:08.325512 2656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:08.325583 2656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:08.325618 2656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:42:08 dist-test-slave-gkw7 krb5kdc[2060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264928, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:08.333709 2656 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:45175
I20260809 08:42:08.333814 2843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:45175 every 8 connection(s)
I20260809 08:42:08.334136 2656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:42:08.336803 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.326025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:43465 (local address 127.31.63.126:33763)
0809 08:42:08.326102 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.326105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.326326 (+ 221us) server_negotiation.cc:407] Connection header received
0809 08:42:08.326822 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.326824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.326838 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.326920 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:08.327596 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.328463 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.330103 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.330427 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.330497 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.334058 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:08.334059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:08.334059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:08.334067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:08.335589 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.336008 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.336009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.336009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.336060 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.336283 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.336283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.336284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.336389 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:08.336439 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.336673 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.336737 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 08:42:08.337015 2686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.326004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33763 (local address 127.31.63.67:43465)
0809 08:42:08.326193 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.326234 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.326729 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.327027 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.327031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.327320 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:08.327501 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.327506 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.328573 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.328574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.330003 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.330007 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.330527 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.330528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.330707 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.331237 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:08.331248 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:08.333952 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:08.335672 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.335678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.335680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.335937 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.336119 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.336120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.336120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.336227 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.336442 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:08.336447 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:08.336562 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.336644 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.336772 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":125,"threads_started":1}
I20260809 08:42:08.337579 2844 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33763
I20260809 08:42:08.337687 2844 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:08.337891 2844 heartbeater.cc:511] Master 127.31.63.126:33763 requested a full tablet report, sending...
I20260809 08:42:08.338001 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 2656
I20260809 08:42:08.338073 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:08.338254 31997 external_mini_cluster.cc:1428] Setting key fa84d26e342a1e132d9d1d752cfeab8e
I20260809 08:42:08.338747 2127 ts_manager.cc:194] Registered new tserver with Master: dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175)
I20260809 08:42:08.339093 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:42:08.339295 2127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:43465
I20260809 08:42:08.344687 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.338725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:55971 (local address 127.31.63.126:33763)
0809 08:42:08.338818 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.338821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.338942 (+ 121us) server_negotiation.cc:407] Connection header received
0809 08:42:08.338999 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.339000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.339010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.339060 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:08.339614 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.340066 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.340804 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.341002 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.341076 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.342251 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:08.342253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:08.342253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:08.342259 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:08.343638 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.344038 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.344039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.344039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.344085 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.344265 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.344266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.344266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.344353 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:08.344399 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.344573 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.344628 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 08:42:08.344775 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.338686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33763 (local address 127.31.63.67:55971)
0809 08:42:08.338862 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.338875 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.338937 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.339068 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.339069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.339288 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:08.339356 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.339359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.340168 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.340170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.340704 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.340705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.341077 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.341077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.341160 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.341739 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:08.341745 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:08.342168 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:08.343727 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.343728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.343728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.343960 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.344128 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.344129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.344130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.344207 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.344418 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:08.344419 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:08.344475 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.344558 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.344673 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
Aug 09 08:42:08 dist-test-slave-gkw7 krb5kdc[2060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:42:08 dist-test-slave-gkw7 krb5kdc[2060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:08.363889 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.356998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41480 (local address 127.31.63.126:33763)
0809 08:42:08.357069 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.357072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.357082 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:08.357119 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.357120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.357130 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.357189 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:08.357893 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.358387 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.359139 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.359341 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.359416 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.361582 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:08.361584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:08.361584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:08.361594 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:08.362756 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.363104 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.363106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.363106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.363161 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.363372 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.363373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.363374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.363482 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:08.363594 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.363700 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.363783 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:42:08.365391 2127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:42:08.367043 2127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:08.373502 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.369413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:43571 (local address 127.0.0.1:59206)
0809 08:42:08.369632 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.369677 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.369749 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.370099 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.370102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.370106 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:08.370214 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.370216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.371610 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.371616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.372400 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.372402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.373189 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.373190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.373276 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.373291 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.373369 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.373423 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":6,"thread_start_us":59,"threads_started":1}
I20260809 08:42:08.373567 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.369640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.31.63.66:43571)
0809 08:42:08.369913 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.369918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.369929 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:42:08.369971 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.369973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.369989 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.370292 (+ 303us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:08.370359 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.371494 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.372497 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.373109 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.373200 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.373261 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.373397 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.373437 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":104,"threads_started":1}
I20260809 08:42:08.374251 2876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.369414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35985 (local address 127.0.0.1:58592)
0809 08:42:08.370155 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.370165 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.370367 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.370571 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.370572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.370574 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:08.370631 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.370633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.371469 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.371470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.372793 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.372796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.373976 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.373977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.374040 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.374045 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.374127 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.374169 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":168,"thread_start_us":114,"threads_started":1}
I20260809 08:42:08.374456 2877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.369414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:45175 (local address 127.0.0.1:47076)
0809 08:42:08.369675 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.369684 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.369749 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.370099 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.370102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.370105 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:08.370201 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.370203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.371586 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.371587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.373020 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.373022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.374241 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.374242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.374309 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.374313 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.374347 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.374389 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":87,"thread_start_us":97,"threads_started":1}
I20260809 08:42:08.374346 2539 tablet_service.cc:1467] Processing CreateTablet for tablet 2a8d32f37d134eea936f7839bae2aa27 (DEFAULT_TABLE table=test-table [id=51507a2523a647ea805673f4d962ecc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:08.374491 2883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.369861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58592 (local address 127.31.63.65:35985)
0809 08:42:08.370098 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.370104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.370329 (+ 225us) server_negotiation.cc:407] Connection header received
0809 08:42:08.370468 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.370470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.370486 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.370576 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:08.370690 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.371379 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.372886 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.373867 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.373972 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.374033 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.374147 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.374190 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":112,"threads_started":1}
I20260809 08:42:08.374763 2539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a8d32f37d134eea936f7839bae2aa27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:08.374832 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.369611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47076 (local address 127.31.63.67:45175)
0809 08:42:08.369915 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.369919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.369929 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:08.369971 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.369973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.369989 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.370290 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:08.370359 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.371484 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.373103 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.374135 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.374580 (+ 445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.374651 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.374680 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.374741 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":82,"threads_started":1}
I20260809 08:42:08.375262 2329 tablet_service.cc:1467] Processing CreateTablet for tablet 2a8d32f37d134eea936f7839bae2aa27 (DEFAULT_TABLE table=test-table [id=51507a2523a647ea805673f4d962ecc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:08.375468 2743 tablet_service.cc:1467] Processing CreateTablet for tablet 2a8d32f37d134eea936f7839bae2aa27 (DEFAULT_TABLE table=test-table [id=51507a2523a647ea805673f4d962ecc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:08.375612 2329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a8d32f37d134eea936f7839bae2aa27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:08.375784 2743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a8d32f37d134eea936f7839bae2aa27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:08.379045 2885 tablet_bootstrap.cc:492] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: Bootstrap starting.
I20260809 08:42:08.379384 2886 tablet_bootstrap.cc:492] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: Bootstrap starting.
I20260809 08:42:08.379545 2884 tablet_bootstrap.cc:492] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: Bootstrap starting.
I20260809 08:42:08.379985 2885 tablet_bootstrap.cc:654] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:08.380090 2886 tablet_bootstrap.cc:654] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:08.380185 2884 tablet_bootstrap.cc:654] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:08.380374 2885 log.cc:824] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:08.380475 2886 log.cc:824] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:08.380522 2884 log.cc:824] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:08.381275 2884 tablet_bootstrap.cc:492] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: No bootstrap required, opened a new log
I20260809 08:42:08.381362 2884 ts_tablet_manager.cc:1397] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:42:08.381472 2885 tablet_bootstrap.cc:492] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: No bootstrap required, opened a new log
I20260809 08:42:08.381582 2885 ts_tablet_manager.cc:1397] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:42:08.382875 2884 raft_consensus.cc:348] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } }
I20260809 08:42:08.383029 2884 raft_consensus.cc:374] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:08.382966 2885 raft_consensus.cc:348] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } }
I20260809 08:42:08.383078 2884 raft_consensus.cc:729] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7fb0ad9c8a34fb09ee3734ecf600956, State: Initialized, Role: FOLLOWER
I20260809 08:42:08.383121 2885 raft_consensus.cc:374] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:08.383162 2885 raft_consensus.cc:729] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc23d7916aec4c19bd1898e1f1144b13, State: Initialized, Role: FOLLOWER
I20260809 08:42:08.383201 2884 consensus_queue.cc:260] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } }
I20260809 08:42:08.383283 2885 consensus_queue.cc:260] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } }
I20260809 08:42:08.383491 2884 ts_tablet_manager.cc:1428] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:08.383522 2643 heartbeater.cc:503] Master 127.31.63.126:33763 was elected leader, sending a full tablet report...
I20260809 08:42:08.383565 2885 ts_tablet_manager.cc:1428] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:08.383749 2844 heartbeater.cc:503] Master 127.31.63.126:33763 was elected leader, sending a full tablet report...
I20260809 08:42:08.383944 2886 tablet_bootstrap.cc:492] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: No bootstrap required, opened a new log
I20260809 08:42:08.384020 2886 ts_tablet_manager.cc:1397] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 08:42:08.385737 2886 raft_consensus.cc:348] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } }
I20260809 08:42:08.385859 2886 raft_consensus.cc:374] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:08.385880 2886 raft_consensus.cc:729] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e79bd558378c443cb7139a70215a0c33, State: Initialized, Role: FOLLOWER
I20260809 08:42:08.386006 2886 consensus_queue.cc:260] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } }
I20260809 08:42:08.386270 2886 ts_tablet_manager.cc:1428] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:42:08.386353 2430 heartbeater.cc:503] Master 127.31.63.126:33763 was elected leader, sending a full tablet report...
W20260809 08:42:08.425915 2644 tablet.cc:2369] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:42:08.514112 2431 tablet.cc:2369] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:42:08.553956 2894 raft_consensus.cc:482] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:08.554159 2894 raft_consensus.cc:504] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } }
I20260809 08:42:08.554610 2894 leader_election.cc:288] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175), e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985)
I20260809 08:42:08.557690 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.554810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35985 (local address 127.31.63.66:56189)
0809 08:42:08.555008 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.555019 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.555096 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.555269 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.555269 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.555271 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:08.555363 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.555365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.556042 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.556043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.556792 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.556793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.557377 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.557378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.557453 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.557458 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.557535 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.557596 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":140,"threads_started":1}
I20260809 08:42:08.557669 2883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.554888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:56189 (local address 127.31.63.65:35985)
0809 08:42:08.554937 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.554940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.555086 (+ 146us) server_negotiation.cc:407] Connection header received
0809 08:42:08.555153 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.555154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.555164 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.555260 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:08.555442 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.555962 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.556891 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.557292 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.557377 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.557387 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.557563 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.557599 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:42:08.557778 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.554930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:60213 (local address 127.31.63.67:45175)
0809 08:42:08.554980 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.554982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.554992 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:08.555029 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.555030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.555038 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.555117 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:08.555302 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.555866 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.556821 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.557368 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.557453 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.557462 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.557674 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.557711 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:42:08.557772 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.554781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:45175 (local address 127.31.63.66:60213)
0809 08:42:08.554843 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.554856 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.554900 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.555124 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.555126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.555127 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:08.555221 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.555223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.555947 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.555948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.556727 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.556729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.557452 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.557453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.557587 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.557590 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.557655 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.557700 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 08:42:08.558168 2773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8d32f37d134eea936f7839bae2aa27" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc23d7916aec4c19bd1898e1f1144b13" is_pre_election: true
I20260809 08:42:08.558166 2361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8d32f37d134eea936f7839bae2aa27" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79bd558378c443cb7139a70215a0c33" is_pre_election: true
I20260809 08:42:08.558353 2361 raft_consensus.cc:2454] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 0.
I20260809 08:42:08.558357 2773 raft_consensus.cc:2454] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 0.
I20260809 08:42:08.558632 2471 leader_election.cc:302] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7fb0ad9c8a34fb09ee3734ecf600956, e79bd558378c443cb7139a70215a0c33; no voters:
I20260809 08:42:08.558790 2894 raft_consensus.cc:2781] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:08.558856 2894 raft_consensus.cc:482] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:08.558884 2894 raft_consensus.cc:3037] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:08.559826 2894 raft_consensus.cc:504] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } }
I20260809 08:42:08.560003 2894 leader_election.cc:288] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 election: Requested vote from peers dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175), e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985)
I20260809 08:42:08.560169 2773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8d32f37d134eea936f7839bae2aa27" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc23d7916aec4c19bd1898e1f1144b13"
I20260809 08:42:08.560226 2361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a8d32f37d134eea936f7839bae2aa27" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79bd558378c443cb7139a70215a0c33"
I20260809 08:42:08.560264 2773 raft_consensus.cc:3037] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:08.560333 2361 raft_consensus.cc:3037] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:08.561230 2773 raft_consensus.cc:2454] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 1.
I20260809 08:42:08.561297 2361 raft_consensus.cc:2454] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 1.
I20260809 08:42:08.561447 2465 leader_election.cc:302] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7fb0ad9c8a34fb09ee3734ecf600956, dc23d7916aec4c19bd1898e1f1144b13; no voters:
I20260809 08:42:08.561590 2894 raft_consensus.cc:2781] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:08.561744 2894 raft_consensus.cc:686] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 LEADER]: Becoming Leader. State: Replica: a7fb0ad9c8a34fb09ee3734ecf600956, State: Running, Role: LEADER
I20260809 08:42:08.561863 2894 consensus_queue.cc:237] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } }
I20260809 08:42:08.562742 2128 catalog_manager.cc:5495] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } health_report { overall_health: UNKNOWN } } }
W20260809 08:42:08.584935 2850 tablet.cc:2369] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 08:42:08 dist-test-slave-gkw7 krb5kdc[2060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264927, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:08.625356 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.618461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41494 (local address 127.31.63.126:33763)
0809 08:42:08.618557 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.618560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.618732 (+ 172us) server_negotiation.cc:407] Connection header received
0809 08:42:08.618807 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.618808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.618819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.618865 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:08.619360 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.619930 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.620634 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.620843 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.620932 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.622976 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:08.622977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:08.622977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:08.622985 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:08.624151 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.624537 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.624539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.624539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.624599 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:08.624798 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:08.624799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:08.624799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:08.624917 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:08.624995 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.625153 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.625240 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 08:42:08.625424 2914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.618437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33763 (local address 127.0.0.1:41494)
0809 08:42:08.618641 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.618651 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.618741 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.618868 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.618869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.619217 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:08.619268 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.619270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.620013 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.620014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.620546 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.620548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.620933 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.620934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.620998 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.621478 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:08.621483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:08.622881 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:08.624255 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.624263 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.624266 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.624460 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.624673 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:08.624674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:08.624674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:08.624735 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:08.624985 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:08.624993 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:08.625042 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.625130 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.625297 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":67,"thread_start_us":98,"threads_started":1}
I20260809 08:42:08.627434 2128 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:41494:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 08:42:08.627681 2128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:08.629276 2329 tablet_service.cc:1467] Processing CreateTablet for tablet 30587ba8e4224358bf88687281635853 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4be8aef5506a42b5aaff9bc6ed439b52]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:42:08.629293 2743 tablet_service.cc:1467] Processing CreateTablet for tablet 30587ba8e4224358bf88687281635853 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4be8aef5506a42b5aaff9bc6ed439b52]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:42:08.629303 2539 tablet_service.cc:1467] Processing CreateTablet for tablet 30587ba8e4224358bf88687281635853 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4be8aef5506a42b5aaff9bc6ed439b52]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:42:08.629470 2743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30587ba8e4224358bf88687281635853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:08.629470 2329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30587ba8e4224358bf88687281635853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:08.629491 2539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30587ba8e4224358bf88687281635853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:08.630957 2884 tablet_bootstrap.cc:492] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: Bootstrap starting.
I20260809 08:42:08.631071 2885 tablet_bootstrap.cc:492] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: Bootstrap starting.
I20260809 08:42:08.631084 2886 tablet_bootstrap.cc:492] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: Bootstrap starting.
I20260809 08:42:08.631455 2884 tablet_bootstrap.cc:654] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:08.631517 2886 tablet_bootstrap.cc:654] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:08.631661 2885 tablet_bootstrap.cc:654] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:08.632114 2884 tablet_bootstrap.cc:492] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: No bootstrap required, opened a new log
I20260809 08:42:08.632177 2884 ts_tablet_manager.cc:1397] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:08.632325 2885 tablet_bootstrap.cc:492] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: No bootstrap required, opened a new log
I20260809 08:42:08.632380 2885 ts_tablet_manager.cc:1397] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:08.632350 2884 raft_consensus.cc:348] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.632479 2884 raft_consensus.cc:374] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:08.632508 2884 raft_consensus.cc:729] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7fb0ad9c8a34fb09ee3734ecf600956, State: Initialized, Role: FOLLOWER
I20260809 08:42:08.632542 2885 raft_consensus.cc:348] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.632560 2884 consensus_queue.cc:260] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.632611 2885 raft_consensus.cc:374] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:08.632635 2885 raft_consensus.cc:729] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc23d7916aec4c19bd1898e1f1144b13, State: Initialized, Role: FOLLOWER
I20260809 08:42:08.632674 2884 ts_tablet_manager.cc:1428] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:08.632685 2885 consensus_queue.cc:260] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.632740 2894 tablet_replica.cc:439] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } }
I20260809 08:42:08.632804 2885 ts_tablet_manager.cc:1428] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:08.632839 2894 tablet_replica.cc:442] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:08.632864 2897 tablet_replica.cc:439] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } }
I20260809 08:42:08.632942 2897 tablet_replica.cc:442] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:08.633417 2886 tablet_bootstrap.cc:492] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: No bootstrap required, opened a new log
I20260809 08:42:08.633472 2886 ts_tablet_manager.cc:1397] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:42:08.633665 2886 raft_consensus.cc:348] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.633764 2886 raft_consensus.cc:374] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:08.633797 2886 raft_consensus.cc:729] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e79bd558378c443cb7139a70215a0c33, State: Initialized, Role: FOLLOWER
I20260809 08:42:08.633868 2886 consensus_queue.cc:260] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.633967 2886 ts_tablet_manager.cc:1428] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:42:08.634166 2899 tablet_replica.cc:439] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } }
I20260809 08:42:08.634287 2899 tablet_replica.cc:442] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:08.697355 2894 raft_consensus.cc:482] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:08.697467 2894 raft_consensus.cc:504] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.697705 2894 leader_election.cc:288] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985), dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175)
I20260809 08:42:08.697932 2361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30587ba8e4224358bf88687281635853" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79bd558378c443cb7139a70215a0c33" is_pre_election: true
I20260809 08:42:08.698045 2361 raft_consensus.cc:2454] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 0.
I20260809 08:42:08.698045 2773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30587ba8e4224358bf88687281635853" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc23d7916aec4c19bd1898e1f1144b13" is_pre_election: true
I20260809 08:42:08.698204 2471 leader_election.cc:302] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7fb0ad9c8a34fb09ee3734ecf600956, e79bd558378c443cb7139a70215a0c33; no voters:
I20260809 08:42:08.698213 2773 raft_consensus.cc:2454] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 0.
I20260809 08:42:08.698316 2894 raft_consensus.cc:2781] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:08.698357 2894 raft_consensus.cc:482] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:08.698381 2894 raft_consensus.cc:3037] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:08.699105 2894 raft_consensus.cc:504] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.699239 2894 leader_election.cc:288] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 election: Requested vote from peers e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985), dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175)
I20260809 08:42:08.699421 2361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30587ba8e4224358bf88687281635853" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79bd558378c443cb7139a70215a0c33"
I20260809 08:42:08.699450 2773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30587ba8e4224358bf88687281635853" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc23d7916aec4c19bd1898e1f1144b13"
I20260809 08:42:08.699520 2773 raft_consensus.cc:3037] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:08.699523 2361 raft_consensus.cc:3037] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:08.700264 2773 raft_consensus.cc:2454] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 1.
I20260809 08:42:08.700454 2361 raft_consensus.cc:2454] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 1.
I20260809 08:42:08.700476 2465 leader_election.cc:302] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7fb0ad9c8a34fb09ee3734ecf600956, dc23d7916aec4c19bd1898e1f1144b13; no voters:
I20260809 08:42:08.700563 2894 raft_consensus.cc:2781] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:08.700639 2894 raft_consensus.cc:686] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 LEADER]: Becoming Leader. State: Replica: a7fb0ad9c8a34fb09ee3734ecf600956, State: Running, Role: LEADER
I20260809 08:42:08.700703 2894 consensus_queue.cc:237] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.701012 2903 tablet_replica.cc:439] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: TxnStatusTablet state changed. Reason: New leader a7fb0ad9c8a34fb09ee3734ecf600956. Latest consensus state: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } }
I20260809 08:42:08.701100 2903 tablet_replica.cc:442] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:42:08.701292 2919 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 08:42:08.701272 2127 catalog_manager.cc:5495] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } health_report { overall_health: UNKNOWN } } }
I20260809 08:42:08.701427 2919 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 08:42:08.709079 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.706629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.31.63.66:43571)
0809 08:42:08.706694 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.706696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.706705 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:42:08.706794 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.706795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.706813 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.706879 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:08.706986 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.707473 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.708238 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.708447 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.708534 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.708744 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:08.708842 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:08.708935 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.708987 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.709008 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:42:08.709115 2914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.706555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:43571 (local address 127.0.0.1:59222)
0809 08:42:08.706632 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.706667 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.706724 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.706883 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.706884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.706885 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:42:08.706937 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.706939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.707559 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.707560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.708144 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.708145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.708535 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.708536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.708606 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.708631 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:42:08.708911 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:42:08.708921 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.708985 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.709035 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 08:42:08.710930 2773 raft_consensus.cc:1261] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Refusing update from remote peer a7fb0ad9c8a34fb09ee3734ecf600956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:08.710930 2361 raft_consensus.cc:1261] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Refusing update from remote peer a7fb0ad9c8a34fb09ee3734ecf600956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:08.711270 2903 consensus_queue.cc:1035] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:08.711453 2903 consensus_queue.cc:1035] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:08.712631 2899 tablet_replica.cc:439] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: TxnStatusTablet state changed. Reason: New leader a7fb0ad9c8a34fb09ee3734ecf600956. Latest consensus state: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } }
I20260809 08:42:08.712738 2899 tablet_replica.cc:442] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:08.712663 2897 tablet_replica.cc:439] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: TxnStatusTablet state changed. Reason: New leader a7fb0ad9c8a34fb09ee3734ecf600956. Latest consensus state: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } }
I20260809 08:42:08.712777 2897 tablet_replica.cc:442] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:08.714149 2903 tablet_replica.cc:439] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } }
I20260809 08:42:08.714229 2903 tablet_replica.cc:442] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:42:08.714313 2925 mvcc.cc:205] Tried to move back new op lower bound from 7316541147997466624 to 7316541147958456320. Current Snapshot: MvccSnapshot[applied={T|T < 7316541147997466624}]
I20260809 08:42:08.714448 2897 tablet_replica.cc:439] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } }
I20260809 08:42:08.714532 2897 tablet_replica.cc:442] T 30587ba8e4224358bf88687281635853 P dc23d7916aec4c19bd1898e1f1144b13: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:08.715273 2928 mvcc.cc:205] Tried to move back new op lower bound from 7316541147997466624 to 7316541147958456320. Current Snapshot: MvccSnapshot[applied={T|T < 7316541147997466624}]
I20260809 08:42:08.715605 2903 tablet_replica.cc:439] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } }
I20260809 08:42:08.715687 2903 tablet_replica.cc:442] T 30587ba8e4224358bf88687281635853 P a7fb0ad9c8a34fb09ee3734ecf600956: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:42:08.716050 2899 tablet_replica.cc:439] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } } }
I20260809 08:42:08.716140 2899 tablet_replica.cc:442] T 30587ba8e4224358bf88687281635853 P e79bd558378c443cb7139a70215a0c33: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:42:08.716264 2926 mvcc.cc:205] Tried to move back new op lower bound from 7316541147997466624 to 7316541147958456320. Current Snapshot: MvccSnapshot[applied={T|T < 7316541147997466624}]
I20260809 08:42:08.719470 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.716688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.31.63.66:43571)
0809 08:42:08.716727 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.716728 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.716735 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:42:08.716756 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.716756 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.716784 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.716836 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:08.717023 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.717832 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.718684 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.718989 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.719062 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.719221 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:08.719258 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:08.719312 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.719375 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.719399 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:42:08.725266 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.722077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:35821 (local address 127.31.63.66:43571)
0809 08:42:08.722121 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:08.722123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:08.722270 (+ 147us) server_negotiation.cc:407] Connection header received
0809 08:42:08.722298 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:08.722299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:08.722309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:08.722377 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:08.722475 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.723185 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.724442 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.724768 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.724842 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.725005 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:08.725037 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:08.725090 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:08.725147 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 08:42:08.725171 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":23}
I20260809 08:42:08.725272 2942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:08.722020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:43571 (local address 127.31.63.66:35821)
0809 08:42:08.722182 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:08.722207 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:08.722256 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:08.722380 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:08.722381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:08.722381 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:42:08.722427 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.722428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.723261 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.723261 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.724351 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:08.724354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:08.724849 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:08.724849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:08.724897 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:08.724911 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:42:08.725099 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:42:08.725100 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:42:08.725131 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 08:42:08.725169 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260809 08:42:08.727890 2361 raft_consensus.cc:1261] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Refusing update from remote peer a7fb0ad9c8a34fb09ee3734ecf600956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:08.727926 2773 raft_consensus.cc:1261] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Refusing update from remote peer a7fb0ad9c8a34fb09ee3734ecf600956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:08.728123 2894 consensus_queue.cc:1035] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:08.728231 2903 consensus_queue.cc:1035] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:08.746395 2128 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:08.746526 2128 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:08.747346 2128 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5: Sending DeleteTablet for 3 replicas of tablet 2a8d32f37d134eea936f7839bae2aa27
I20260809 08:42:08.747726 2329 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a8d32f37d134eea936f7839bae2aa27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:08 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:58592
I20260809 08:42:08.747727 2743 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a8d32f37d134eea936f7839bae2aa27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:08 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:47076
I20260809 08:42:08.747838 2539 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a8d32f37d134eea936f7839bae2aa27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:08 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:59206
I20260809 08:42:08.748229 2952 tablet_replica.cc:330] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: stopping tablet replica
I20260809 08:42:08.748225 2128 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 08:42:08.748358 2952 raft_consensus.cc:2229] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 LEADER]: Raft consensus shutting down.
W20260809 08:42:08.748370 2128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:08.748471 2952 raft_consensus.cc:2258] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:08.748710 2951 tablet_replica.cc:330] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: stopping tablet replica
I20260809 08:42:08.748885 2951 raft_consensus.cc:2229] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:08.748999 2951 raft_consensus.cc:2258] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:08.748991 2952 ts_tablet_manager.cc:1905] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:08.749447 2329 tablet_service.cc:1467] Processing CreateTablet for tablet 6248863d0da9468192300e16786abd93 (DEFAULT_TABLE table=test-table [id=1a2b8dd8ddfa4355a723de52b52e1705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:08.749591 2329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6248863d0da9468192300e16786abd93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:08.749792 2539 tablet_service.cc:1467] Processing CreateTablet for tablet 6248863d0da9468192300e16786abd93 (DEFAULT_TABLE table=test-table [id=1a2b8dd8ddfa4355a723de52b52e1705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:08.749902 2539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6248863d0da9468192300e16786abd93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:08.750943 2952 ts_tablet_manager.cc:1918] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:42:08.751014 2952 log.cc:1196] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/2a8d32f37d134eea936f7839bae2aa27
I20260809 08:42:08.751058 2743 tablet_service.cc:1467] Processing CreateTablet for tablet 6248863d0da9468192300e16786abd93 (DEFAULT_TABLE table=test-table [id=1a2b8dd8ddfa4355a723de52b52e1705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:08.751163 2884 tablet_bootstrap.cc:492] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956: Bootstrap starting.
I20260809 08:42:08.751188 2743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6248863d0da9468192300e16786abd93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:08.751297 2952 ts_tablet_manager.cc:1939] T 2a8d32f37d134eea936f7839bae2aa27 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting consensus metadata
I20260809 08:42:08.751578 2884 tablet_bootstrap.cc:654] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:08.752238 2884 tablet_bootstrap.cc:492] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956: No bootstrap required, opened a new log
I20260809 08:42:08.752278 2884 ts_tablet_manager.cc:1397] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:08.752266 2108 catalog_manager.cc:4841] TS a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66:43571): tablet 2a8d32f37d134eea936f7839bae2aa27 (table test-table [id=51507a2523a647ea805673f4d962ecc7]) successfully deleted
I20260809 08:42:08.752419 2884 raft_consensus.cc:348] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.752494 2884 raft_consensus.cc:374] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:08.752516 2884 raft_consensus.cc:729] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7fb0ad9c8a34fb09ee3734ecf600956, State: Initialized, Role: FOLLOWER
I20260809 08:42:08.752558 2884 consensus_queue.cc:260] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.752677 2884 ts_tablet_manager.cc:1428] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:08.752676 2886 tablet_bootstrap.cc:492] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33: Bootstrap starting.
I20260809 08:42:08.753078 2886 tablet_bootstrap.cc:654] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:08.753459 2885 tablet_bootstrap.cc:492] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13: Bootstrap starting.
I20260809 08:42:08.753904 2886 tablet_bootstrap.cc:492] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33: No bootstrap required, opened a new log
I20260809 08:42:08.753944 2886 ts_tablet_manager.cc:1397] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:08.754084 2886 raft_consensus.cc:348] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.754160 2886 raft_consensus.cc:374] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:08.754177 2886 raft_consensus.cc:729] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e79bd558378c443cb7139a70215a0c33, State: Initialized, Role: FOLLOWER
I20260809 08:42:08.754217 2886 consensus_queue.cc:260] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.754293 2886 ts_tablet_manager.cc:1428] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:08.754951 2885 tablet_bootstrap.cc:654] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:08.755628 2885 tablet_bootstrap.cc:492] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13: No bootstrap required, opened a new log
I20260809 08:42:08.755699 2885 ts_tablet_manager.cc:1397] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:42:08.755863 2885 raft_consensus.cc:348] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.755918 2885 raft_consensus.cc:374] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:08.755930 2885 raft_consensus.cc:729] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc23d7916aec4c19bd1898e1f1144b13, State: Initialized, Role: FOLLOWER
I20260809 08:42:08.755981 2885 consensus_queue.cc:260] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.756076 2885 ts_tablet_manager.cc:1428] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:08.756520 2951 ts_tablet_manager.cc:1905] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:08.757951 2950 tablet_replica.cc:330] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: stopping tablet replica
I20260809 08:42:08.757985 2951 ts_tablet_manager.cc:1918] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:42:08.758046 2951 log.cc:1196] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/2a8d32f37d134eea936f7839bae2aa27
I20260809 08:42:08.758085 2950 raft_consensus.cc:2229] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:08.758155 2950 raft_consensus.cc:2258] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:08.758329 2951 ts_tablet_manager.cc:1939] T 2a8d32f37d134eea936f7839bae2aa27 P e79bd558378c443cb7139a70215a0c33: Deleting consensus metadata
I20260809 08:42:08.758601 2950 ts_tablet_manager.cc:1905] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:08.758774 2110 catalog_manager.cc:4841] TS e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985): tablet 2a8d32f37d134eea936f7839bae2aa27 (table test-table [id=51507a2523a647ea805673f4d962ecc7]) successfully deleted
I20260809 08:42:08.759830 2950 ts_tablet_manager.cc:1918] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:42:08.759892 2950 log.cc:1196] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/2a8d32f37d134eea936f7839bae2aa27
I20260809 08:42:08.760073 2950 ts_tablet_manager.cc:1939] T 2a8d32f37d134eea936f7839bae2aa27 P dc23d7916aec4c19bd1898e1f1144b13: Deleting consensus metadata
I20260809 08:42:08.760527 2106 catalog_manager.cc:4841] TS dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175): tablet 2a8d32f37d134eea936f7839bae2aa27 (table test-table [id=51507a2523a647ea805673f4d962ecc7]) successfully deleted
I20260809 08:42:08.785035 2947 raft_consensus.cc:482] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:08.785159 2947 raft_consensus.cc:504] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.785421 2947 leader_election.cc:288] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985), dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175)
I20260809 08:42:08.785573 2361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6248863d0da9468192300e16786abd93" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79bd558378c443cb7139a70215a0c33" is_pre_election: true
I20260809 08:42:08.785575 2773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6248863d0da9468192300e16786abd93" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc23d7916aec4c19bd1898e1f1144b13" is_pre_election: true
I20260809 08:42:08.785658 2361 raft_consensus.cc:2454] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 0.
I20260809 08:42:08.785683 2773 raft_consensus.cc:2454] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 0.
I20260809 08:42:08.785851 2465 leader_election.cc:302] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7fb0ad9c8a34fb09ee3734ecf600956, dc23d7916aec4c19bd1898e1f1144b13; no voters:
I20260809 08:42:08.785946 2947 raft_consensus.cc:2781] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:08.785977 2947 raft_consensus.cc:482] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:08.785997 2947 raft_consensus.cc:3037] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:08.786996 2947 raft_consensus.cc:504] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.787210 2947 leader_election.cc:288] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 election: Requested vote from peers e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985), dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175)
I20260809 08:42:08.787292 2773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6248863d0da9468192300e16786abd93" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc23d7916aec4c19bd1898e1f1144b13"
I20260809 08:42:08.787330 2361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6248863d0da9468192300e16786abd93" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79bd558378c443cb7139a70215a0c33"
I20260809 08:42:08.787375 2773 raft_consensus.cc:3037] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:08.787418 2361 raft_consensus.cc:3037] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:08.788089 2361 raft_consensus.cc:2454] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 1.
I20260809 08:42:08.788168 2773 raft_consensus.cc:2454] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 1.
I20260809 08:42:08.788283 2471 leader_election.cc:302] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7fb0ad9c8a34fb09ee3734ecf600956, e79bd558378c443cb7139a70215a0c33; no voters:
I20260809 08:42:08.788398 2947 raft_consensus.cc:2781] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:08.788446 2947 raft_consensus.cc:686] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 LEADER]: Becoming Leader. State: Replica: a7fb0ad9c8a34fb09ee3734ecf600956, State: Running, Role: LEADER
I20260809 08:42:08.788501 2947 consensus_queue.cc:237] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:08.789120 2127 catalog_manager.cc:5495] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } health_report { overall_health: UNKNOWN } } }
I20260809 08:42:08.801370 2773 raft_consensus.cc:1261] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Refusing update from remote peer a7fb0ad9c8a34fb09ee3734ecf600956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:08.801450 2361 raft_consensus.cc:1261] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Refusing update from remote peer a7fb0ad9c8a34fb09ee3734ecf600956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:08.801637 2903 consensus_queue.cc:1035] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:08.801764 2947 consensus_queue.cc:1035] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 08:42:08.807008 2925 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 08:42:10.328352 2127 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:10.328495 2127 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:10.329411 2127 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5: Sending DeleteTablet for 3 replicas of tablet 6248863d0da9468192300e16786abd93
I20260809 08:42:10.329758 2743 tablet_service.cc:1514] Processing DeleteTablet for tablet 6248863d0da9468192300e16786abd93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:10 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:47076
I20260809 08:42:10.329797 2329 tablet_service.cc:1514] Processing DeleteTablet for tablet 6248863d0da9468192300e16786abd93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:10 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:58592
I20260809 08:42:10.329813 2538 tablet_service.cc:1514] Processing DeleteTablet for tablet 6248863d0da9468192300e16786abd93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:10 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:59206
I20260809 08:42:10.329990 2990 tablet_replica.cc:330] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33: stopping tablet replica
I20260809 08:42:10.330051 2990 raft_consensus.cc:2229] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:10.330089 2990 raft_consensus.cc:2258] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:10.330385 2990 ts_tablet_manager.cc:1905] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:10.332525 2990 ts_tablet_manager.cc:1918] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:42:10.332603 2990 log.cc:1196] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/6248863d0da9468192300e16786abd93
I20260809 08:42:10.332872 2990 ts_tablet_manager.cc:1939] T 6248863d0da9468192300e16786abd93 P e79bd558378c443cb7139a70215a0c33: Deleting consensus metadata
I20260809 08:42:10.333371 2110 catalog_manager.cc:4841] TS e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985): tablet 6248863d0da9468192300e16786abd93 (table test-table [id=1a2b8dd8ddfa4355a723de52b52e1705]) successfully deleted
I20260809 08:42:10.333675 2989 tablet_replica.cc:330] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13: stopping tablet replica
I20260809 08:42:10.333767 2989 raft_consensus.cc:2229] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:10.333823 2989 raft_consensus.cc:2258] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:10.334105 2989 ts_tablet_manager.cc:1905] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:10.335438 2989 ts_tablet_manager.cc:1918] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:42:10.335533 2989 log.cc:1196] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/6248863d0da9468192300e16786abd93
I20260809 08:42:10.335764 2989 ts_tablet_manager.cc:1939] T 6248863d0da9468192300e16786abd93 P dc23d7916aec4c19bd1898e1f1144b13: Deleting consensus metadata
I20260809 08:42:10.336099 2991 tablet_replica.cc:330] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956: stopping tablet replica
I20260809 08:42:10.336122 2106 catalog_manager.cc:4841] TS dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175): tablet 6248863d0da9468192300e16786abd93 (table test-table [id=1a2b8dd8ddfa4355a723de52b52e1705]) successfully deleted
I20260809 08:42:10.336172 2991 raft_consensus.cc:2229] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:42:10.336243 2991 raft_consensus.cc:2258] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:10.336946 2991 ts_tablet_manager.cc:1905] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:10.338825 2991 ts_tablet_manager.cc:1918] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:42:10.338883 2991 log.cc:1196] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/6248863d0da9468192300e16786abd93
I20260809 08:42:10.339187 2991 ts_tablet_manager.cc:1939] T 6248863d0da9468192300e16786abd93 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting consensus metadata
I20260809 08:42:10.339552 2108 catalog_manager.cc:4841] TS a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66:43571): tablet 6248863d0da9468192300e16786abd93 (table test-table [id=1a2b8dd8ddfa4355a723de52b52e1705]) successfully deleted
Aug 09 08:42:10 dist-test-slave-gkw7 krb5kdc[2060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264930, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:42:10 dist-test-slave-gkw7 krb5kdc[2060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264930, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:10.359362 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.348714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.31.63.126:33763)
0809 08:42:10.348895 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:10.348896 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:10.348903 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:42:10.348924 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:10.348925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:10.348935 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:10.349011 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:10.349698 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.350223 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.351007 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.351219 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.351327 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.356647 (+ 5320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:10.356649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:10.356650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:10.356658 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:10.358170 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:10.358538 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:10.358540 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:10.358540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:10.358603 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:10.358804 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:10.358805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:10.358806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:10.358930 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:10.359041 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:10.359161 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 08:42:10.359250 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260809 08:42:10.360918 2127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:42:10.361066 2127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:10.362504 2329 tablet_service.cc:1467] Processing CreateTablet for tablet 335317b1cfec4178b1c9c3b051cac981 (DEFAULT_TABLE table=test-table [id=dcd556682e834571974c1e4cf30196dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:10.362524 2538 tablet_service.cc:1467] Processing CreateTablet for tablet 335317b1cfec4178b1c9c3b051cac981 (DEFAULT_TABLE table=test-table [id=dcd556682e834571974c1e4cf30196dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:10.362668 2329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335317b1cfec4178b1c9c3b051cac981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:10.362685 2538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335317b1cfec4178b1c9c3b051cac981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:10.363724 3006 tablet_bootstrap.cc:492] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33: Bootstrap starting.
I20260809 08:42:10.363792 2743 tablet_service.cc:1467] Processing CreateTablet for tablet 335317b1cfec4178b1c9c3b051cac981 (DEFAULT_TABLE table=test-table [id=dcd556682e834571974c1e4cf30196dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:10.363922 2743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335317b1cfec4178b1c9c3b051cac981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:10.364051 3007 tablet_bootstrap.cc:492] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956: Bootstrap starting.
I20260809 08:42:10.364085 3006 tablet_bootstrap.cc:654] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:10.364434 3007 tablet_bootstrap.cc:654] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:10.364734 3006 tablet_bootstrap.cc:492] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33: No bootstrap required, opened a new log
I20260809 08:42:10.364794 3006 ts_tablet_manager.cc:1397] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:10.364946 3006 raft_consensus.cc:348] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.365021 3006 raft_consensus.cc:374] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:10.365041 3006 raft_consensus.cc:729] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e79bd558378c443cb7139a70215a0c33, State: Initialized, Role: FOLLOWER
I20260809 08:42:10.365109 3006 consensus_queue.cc:260] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.365388 3016 tablet_bootstrap.cc:492] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13: Bootstrap starting.
I20260809 08:42:10.365314 3006 ts_tablet_manager.cc:1428] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:10.365957 3016 tablet_bootstrap.cc:654] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:10.366003 3007 tablet_bootstrap.cc:492] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956: No bootstrap required, opened a new log
I20260809 08:42:10.366062 3007 ts_tablet_manager.cc:1397] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:42:10.366182 3007 raft_consensus.cc:348] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.366285 3007 raft_consensus.cc:374] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:10.366312 3007 raft_consensus.cc:729] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7fb0ad9c8a34fb09ee3734ecf600956, State: Initialized, Role: FOLLOWER
I20260809 08:42:10.366374 3007 consensus_queue.cc:260] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.366492 3007 ts_tablet_manager.cc:1428] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:10.366747 3016 tablet_bootstrap.cc:492] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13: No bootstrap required, opened a new log
I20260809 08:42:10.366793 3016 ts_tablet_manager.cc:1397] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:10.366971 3016 raft_consensus.cc:348] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.367053 3016 raft_consensus.cc:374] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:10.367077 3016 raft_consensus.cc:729] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc23d7916aec4c19bd1898e1f1144b13, State: Initialized, Role: FOLLOWER
I20260809 08:42:10.367126 3016 consensus_queue.cc:260] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.367304 3016 ts_tablet_manager.cc:1428] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:10.589929 2970 raft_consensus.cc:482] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:10.590030 2970 raft_consensus.cc:504] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.590209 2970 leader_election.cc:288] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985), dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175)
I20260809 08:42:10.590492 2772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335317b1cfec4178b1c9c3b051cac981" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc23d7916aec4c19bd1898e1f1144b13" is_pre_election: true
I20260809 08:42:10.590464 2361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335317b1cfec4178b1c9c3b051cac981" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79bd558378c443cb7139a70215a0c33" is_pre_election: true
I20260809 08:42:10.590607 2772 raft_consensus.cc:2454] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 0.
I20260809 08:42:10.590631 2361 raft_consensus.cc:2454] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 0.
I20260809 08:42:10.590802 2465 leader_election.cc:302] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a7fb0ad9c8a34fb09ee3734ecf600956, dc23d7916aec4c19bd1898e1f1144b13, e79bd558378c443cb7139a70215a0c33; no voters:
I20260809 08:42:10.590925 2970 raft_consensus.cc:2781] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:10.590963 2970 raft_consensus.cc:482] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:10.590991 2970 raft_consensus.cc:3037] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:10.591756 2970 raft_consensus.cc:504] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.591907 2970 leader_election.cc:288] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 election: Requested vote from peers e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985), dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175)
I20260809 08:42:10.592038 2361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335317b1cfec4178b1c9c3b051cac981" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79bd558378c443cb7139a70215a0c33"
I20260809 08:42:10.592150 2361 raft_consensus.cc:3037] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:10.592113 2772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335317b1cfec4178b1c9c3b051cac981" candidate_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc23d7916aec4c19bd1898e1f1144b13"
I20260809 08:42:10.592203 2772 raft_consensus.cc:3037] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:10.592849 2361 raft_consensus.cc:2454] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 1.
I20260809 08:42:10.592864 2772 raft_consensus.cc:2454] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7fb0ad9c8a34fb09ee3734ecf600956 in term 1.
I20260809 08:42:10.593057 2465 leader_election.cc:302] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7fb0ad9c8a34fb09ee3734ecf600956, dc23d7916aec4c19bd1898e1f1144b13; no voters:
I20260809 08:42:10.593159 2970 raft_consensus.cc:2781] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:10.593240 2970 raft_consensus.cc:686] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 LEADER]: Becoming Leader. State: Replica: a7fb0ad9c8a34fb09ee3734ecf600956, State: Running, Role: LEADER
I20260809 08:42:10.593298 2970 consensus_queue.cc:237] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.594038 2128 catalog_manager.cc:5495] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } health_report { overall_health: UNKNOWN } } }
I20260809 08:42:10.653964 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.651410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.31.63.66:43571)
0809 08:42:10.651588 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:10.651590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:10.651596 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:42:10.651615 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:10.651615 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:10.651625 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:10.651698 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:10.651866 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.652393 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.653202 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.653418 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.653495 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.653667 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:10.653709 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:10.653773 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:10.653837 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 08:42:10.653862 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":96,"threads_started":1}
I20260809 08:42:10.656492 2772 raft_consensus.cc:1261] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Refusing update from remote peer a7fb0ad9c8a34fb09ee3734ecf600956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:10.656585 2361 raft_consensus.cc:1261] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Refusing update from remote peer a7fb0ad9c8a34fb09ee3734ecf600956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:10.656723 2947 consensus_queue.cc:1035] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:10.656836 2970 consensus_queue.cc:1035] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:10.666994 2127 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:10.667140 2127 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:10.667865 2127 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5: Sending DeleteTablet for 3 replicas of tablet 335317b1cfec4178b1c9c3b051cac981
I20260809 08:42:10.668133 2329 tablet_service.cc:1514] Processing DeleteTablet for tablet 335317b1cfec4178b1c9c3b051cac981 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:10 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:58592
I20260809 08:42:10.668241 2990 tablet_replica.cc:330] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33: stopping tablet replica
I20260809 08:42:10.668223 2538 tablet_service.cc:1514] Processing DeleteTablet for tablet 335317b1cfec4178b1c9c3b051cac981 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:10 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:59206
I20260809 08:42:10.668216 2743 tablet_service.cc:1514] Processing DeleteTablet for tablet 335317b1cfec4178b1c9c3b051cac981 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:10 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:47076
I20260809 08:42:10.668289 2990 raft_consensus.cc:2229] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:10.668327 2990 raft_consensus.cc:2258] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:10.668334 2991 tablet_replica.cc:330] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956: stopping tablet replica
I20260809 08:42:10.668399 2991 raft_consensus.cc:2229] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:42:10.668457 2991 raft_consensus.cc:2258] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:10.668429 2127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 08:42:10.668534 2989 tablet_replica.cc:330] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13: stopping tablet replica
W20260809 08:42:10.668582 2127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:10.668589 2989 raft_consensus.cc:2229] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:10.668627 2989 raft_consensus.cc:2258] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:10.668619 2990 ts_tablet_manager.cc:1905] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:10.669104 2989 ts_tablet_manager.cc:1905] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:10.669881 2538 tablet_service.cc:1467] Processing CreateTablet for tablet f731567aa68148738227616c0088b1e2 (DEFAULT_TABLE table=test-table [id=943a126aabea4ae5b7e6e5b434d58abc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:10.669883 2329 tablet_service.cc:1467] Processing CreateTablet for tablet f731567aa68148738227616c0088b1e2 (DEFAULT_TABLE table=test-table [id=943a126aabea4ae5b7e6e5b434d58abc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:10.669993 2538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f731567aa68148738227616c0088b1e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:10.669997 2329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f731567aa68148738227616c0088b1e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:10.670332 2990 ts_tablet_manager.cc:1918] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:42:10.670382 2990 log.cc:1196] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/335317b1cfec4178b1c9c3b051cac981
I20260809 08:42:10.670590 2990 ts_tablet_manager.cc:1939] T 335317b1cfec4178b1c9c3b051cac981 P e79bd558378c443cb7139a70215a0c33: Deleting consensus metadata
I20260809 08:42:10.670838 2989 ts_tablet_manager.cc:1918] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:42:10.670900 2989 log.cc:1196] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/335317b1cfec4178b1c9c3b051cac981
I20260809 08:42:10.671132 2989 ts_tablet_manager.cc:1939] T 335317b1cfec4178b1c9c3b051cac981 P dc23d7916aec4c19bd1898e1f1144b13: Deleting consensus metadata
I20260809 08:42:10.671356 3006 tablet_bootstrap.cc:492] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33: Bootstrap starting.
I20260809 08:42:10.671463 3007 tablet_bootstrap.cc:492] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956: Bootstrap starting.
I20260809 08:42:10.671511 2991 ts_tablet_manager.cc:1905] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:10.671816 3007 tablet_bootstrap.cc:654] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:10.672049 3006 tablet_bootstrap.cc:654] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:10.672490 3007 tablet_bootstrap.cc:492] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956: No bootstrap required, opened a new log
I20260809 08:42:10.672535 3007 ts_tablet_manager.cc:1397] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:10.672643 3007 raft_consensus.cc:348] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.672694 3007 raft_consensus.cc:374] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:10.672708 3007 raft_consensus.cc:729] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7fb0ad9c8a34fb09ee3734ecf600956, State: Initialized, Role: FOLLOWER
I20260809 08:42:10.672724 2991 ts_tablet_manager.cc:1918] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:42:10.672807 2991 log.cc:1196] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/335317b1cfec4178b1c9c3b051cac981
I20260809 08:42:10.672981 2991 ts_tablet_manager.cc:1939] T 335317b1cfec4178b1c9c3b051cac981 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting consensus metadata
I20260809 08:42:10.673074 3006 tablet_bootstrap.cc:492] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33: No bootstrap required, opened a new log
I20260809 08:42:10.672742 3007 consensus_queue.cc:260] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.673107 3006 ts_tablet_manager.cc:1397] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:42:10.673189 3007 ts_tablet_manager.cc:1428] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 08:42:10.673250 3006 raft_consensus.cc:348] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.673312 3006 raft_consensus.cc:374] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:10.673355 3006 raft_consensus.cc:729] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e79bd558378c443cb7139a70215a0c33, State: Initialized, Role: FOLLOWER
I20260809 08:42:10.673393 2106 catalog_manager.cc:4841] TS dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175): tablet 335317b1cfec4178b1c9c3b051cac981 (table test-table [id=dcd556682e834571974c1e4cf30196dc]) successfully deleted
I20260809 08:42:10.673405 3006 consensus_queue.cc:260] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.673451 2743 tablet_service.cc:1467] Processing CreateTablet for tablet f731567aa68148738227616c0088b1e2 (DEFAULT_TABLE table=test-table [id=943a126aabea4ae5b7e6e5b434d58abc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:10.673503 3006 ts_tablet_manager.cc:1428] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:10.673576 2743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f731567aa68148738227616c0088b1e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:10.673568 2108 catalog_manager.cc:4841] TS a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66:43571): tablet 335317b1cfec4178b1c9c3b051cac981 (table test-table [id=dcd556682e834571974c1e4cf30196dc]) successfully deleted
I20260809 08:42:10.674721 3016 tablet_bootstrap.cc:492] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13: Bootstrap starting.
I20260809 08:42:10.675133 3016 tablet_bootstrap.cc:654] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:10.675279 2110 catalog_manager.cc:4841] TS e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985): tablet 335317b1cfec4178b1c9c3b051cac981 (table test-table [id=dcd556682e834571974c1e4cf30196dc]) successfully deleted
I20260809 08:42:10.675787 3016 tablet_bootstrap.cc:492] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13: No bootstrap required, opened a new log
I20260809 08:42:10.675859 3016 ts_tablet_manager.cc:1397] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:10.676016 3016 raft_consensus.cc:348] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.676090 3016 raft_consensus.cc:374] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:10.676113 3016 raft_consensus.cc:729] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc23d7916aec4c19bd1898e1f1144b13, State: Initialized, Role: FOLLOWER
I20260809 08:42:10.676167 3016 consensus_queue.cc:260] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.676287 3016 ts_tablet_manager.cc:1428] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:10.836773 3019 raft_consensus.cc:482] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:10.836907 3019 raft_consensus.cc:504] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.837409 3019 leader_election.cc:288] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985), a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66:43571)
I20260809 08:42:10.840664 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.837703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:50437 (local address 127.31.63.66:43571)
0809 08:42:10.837759 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:10.837761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:10.837896 (+ 135us) server_negotiation.cc:407] Connection header received
0809 08:42:10.837987 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:10.837989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:10.838010 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:10.838087 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:10.838266 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.838854 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.839903 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.840299 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.840386 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.840398 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:10.840546 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 08:42:10.840591 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:42:10.840714 3035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.837634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:43571 (local address 127.31.63.67:50437)
0809 08:42:10.837846 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:10.837856 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:10.837907 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:10.838101 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:10.838102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:10.838104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:10.838188 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:10.838191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.838938 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.838939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:10.839807 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:10.839810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.840386 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.840387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:10.840470 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.840474 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:42:10.840540 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 08:42:10.840583 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":66,"threads_started":1}
I20260809 08:42:10.840744 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.837632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35985 (local address 127.31.63.67:35809)
0809 08:42:10.837770 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:10.837782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:10.837849 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:10.838054 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:10.838055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:10.838057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:10.838144 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:10.838146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.838996 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.838997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:10.839818 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:10.839820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.840425 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.840426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:10.840514 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.840517 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:42:10.840585 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 08:42:10.840630 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":131,"threads_started":1}
I20260809 08:42:10.840890 3036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.837705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:35809 (local address 127.31.63.65:35985)
0809 08:42:10.837847 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:10.837849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:10.837860 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:42:10.837930 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:10.837931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:10.837941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:10.838048 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:10.838253 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.838871 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.839915 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.840330 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.840426 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.840435 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:10.840592 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 08:42:10.840631 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
I20260809 08:42:10.841029 2569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f731567aa68148738227616c0088b1e2" candidate_uuid: "dc23d7916aec4c19bd1898e1f1144b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" is_pre_election: true
I20260809 08:42:10.841192 2569 raft_consensus.cc:2454] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc23d7916aec4c19bd1898e1f1144b13 in term 0.
I20260809 08:42:10.841267 2359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f731567aa68148738227616c0088b1e2" candidate_uuid: "dc23d7916aec4c19bd1898e1f1144b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79bd558378c443cb7139a70215a0c33" is_pre_election: true
I20260809 08:42:10.841364 2359 raft_consensus.cc:2454] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc23d7916aec4c19bd1898e1f1144b13 in term 0.
I20260809 08:42:10.841403 2675 leader_election.cc:302] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7fb0ad9c8a34fb09ee3734ecf600956, dc23d7916aec4c19bd1898e1f1144b13; no voters:
I20260809 08:42:10.841557 3019 raft_consensus.cc:2781] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:10.841604 3019 raft_consensus.cc:482] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:10.841626 3019 raft_consensus.cc:3037] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:10.842414 3019 raft_consensus.cc:504] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.842572 3019 leader_election.cc:288] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [CANDIDATE]: Term 1 election: Requested vote from peers e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985), a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66:43571)
I20260809 08:42:10.842751 2569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f731567aa68148738227616c0088b1e2" candidate_uuid: "dc23d7916aec4c19bd1898e1f1144b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956"
I20260809 08:42:10.842749 2359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f731567aa68148738227616c0088b1e2" candidate_uuid: "dc23d7916aec4c19bd1898e1f1144b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79bd558378c443cb7139a70215a0c33"
I20260809 08:42:10.842825 2359 raft_consensus.cc:3037] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:10.842825 2569 raft_consensus.cc:3037] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:10.843667 2359 raft_consensus.cc:2454] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc23d7916aec4c19bd1898e1f1144b13 in term 1.
I20260809 08:42:10.843711 2569 raft_consensus.cc:2454] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc23d7916aec4c19bd1898e1f1144b13 in term 1.
I20260809 08:42:10.843852 2675 leader_election.cc:302] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7fb0ad9c8a34fb09ee3734ecf600956, dc23d7916aec4c19bd1898e1f1144b13; no voters:
I20260809 08:42:10.843952 3019 raft_consensus.cc:2781] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:10.844090 3019 raft_consensus.cc:686] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 LEADER]: Becoming Leader. State: Replica: dc23d7916aec4c19bd1898e1f1144b13, State: Running, Role: LEADER
I20260809 08:42:10.844146 3019 consensus_queue.cc:237] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } }
I20260809 08:42:10.844971 2128 catalog_manager.cc:5495] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67). New cstate: current_term: 1 leader_uuid: "dc23d7916aec4c19bd1898e1f1144b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc23d7916aec4c19bd1898e1f1144b13" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 45175 } health_report { overall_health: HEALTHY } } }
I20260809 08:42:10.859139 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.856619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47082 (local address 127.31.63.67:45175)
0809 08:42:10.856836 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:10.856838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:10.856843 (+ 5us) server_negotiation.cc:407] Connection header received
0809 08:42:10.856879 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:10.856879 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:10.856899 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:10.856963 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:10.857113 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.857653 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.858410 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.858625 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.858675 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.858803 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:10.858909 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:10.858962 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:10.859000 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:42:10.859015 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":165,"threads_started":1}
I20260809 08:42:10.864076 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.861473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:45101 (local address 127.31.63.66:43571)
0809 08:42:10.861574 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:10.861577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:10.861671 (+ 94us) server_negotiation.cc:407] Connection header received
0809 08:42:10.861736 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:10.861737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:10.861756 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:10.861844 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:10.862003 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.862468 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.863303 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.863499 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.863593 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.863812 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:10.863848 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:10.863909 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:10.863973 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 08:42:10.863998 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:42:10.864145 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.861387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:43571 (local address 127.31.63.67:45101)
0809 08:42:10.861598 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:10.861609 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:10.861673 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:10.861850 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:10.861851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:10.861853 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:42:10.861915 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:10.861916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.862551 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.862552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:10.863203 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:10.863205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.863620 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.863621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:10.863695 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.863709 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:42:10.863911 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:42:10.863914 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:42:10.863968 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 08:42:10.864014 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260809 08:42:10.866326 2569 raft_consensus.cc:1261] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Refusing update from remote peer dc23d7916aec4c19bd1898e1f1144b13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:10.866343 2359 raft_consensus.cc:1261] T f731567aa68148738227616c0088b1e2 P e79bd558378c443cb7139a70215a0c33 [term 1 FOLLOWER]: Refusing update from remote peer dc23d7916aec4c19bd1898e1f1144b13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:10.866604 3038 consensus_queue.cc:1035] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e79bd558378c443cb7139a70215a0c33" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:10.866792 3038 consensus_queue.cc:1035] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7fb0ad9c8a34fb09ee3734ecf600956" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 43571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:10.874665 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.872131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:44943 (local address 127.31.63.67:45175)
0809 08:42:10.872199 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:10.872201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:10.872209 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:10.872231 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:10.872232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:10.872242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:10.872298 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:10.872409 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.872881 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.873962 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.874176 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.874250 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.874413 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:10.874448 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:10.874499 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:10.874567 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 08:42:10.874589 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:42:10.874667 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:10.871925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:45175 (local address 127.31.63.66:44943)
0809 08:42:10.872088 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:10.872099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:10.872142 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:10.872302 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:10.872302 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:10.872304 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:42:10.872354 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:10.872356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.872974 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.872974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:10.873878 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:10.873881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:10.874263 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:10.874264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:10.874318 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:10.874322 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:42:10.874506 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:42:10.874507 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:42:10.874547 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 08:42:10.874590 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":109,"threads_started":1}
W20260809 08:42:10.875087 2966 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 08:42:12.399578 2126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:12.399686 2126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:12.400498 2126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98bb4e14d7664a9d8bb774376fdd06e5: Sending DeleteTablet for 3 replicas of tablet f731567aa68148738227616c0088b1e2
I20260809 08:42:12.400782 2538 tablet_service.cc:1514] Processing DeleteTablet for tablet f731567aa68148738227616c0088b1e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:12 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:59206
I20260809 08:42:12.400794 2329 tablet_service.cc:1514] Processing DeleteTablet for tablet f731567aa68148738227616c0088b1e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:12 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:58592
I20260809 08:42:12.400812 2743 tablet_service.cc:1514] Processing DeleteTablet for tablet f731567aa68148738227616c0088b1e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:12 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:47076
I20260809 08:42:12.401003 3083 tablet_replica.cc:330] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956: stopping tablet replica
I20260809 08:42:12.401074 3083 raft_consensus.cc:2229] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:12.401051 3085 tablet_replica.cc:330] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13: stopping tablet replica
I20260809 08:42:12.401115 3083 raft_consensus.cc:2258] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:12.401100 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 2232
I20260809 08:42:12.410310 3083 ts_tablet_manager.cc:1905] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:12.401149 3085 raft_consensus.cc:2229] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:42:12.412774 3083 ts_tablet_manager.cc:1918] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:42:12.412772 3085 raft_consensus.cc:2258] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:12.412833 3083 log.cc:1196] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/f731567aa68148738227616c0088b1e2
I20260809 08:42:12.413033 3083 ts_tablet_manager.cc:1939] T f731567aa68148738227616c0088b1e2 P a7fb0ad9c8a34fb09ee3734ecf600956: Deleting consensus metadata
I20260809 08:42:12.413552 3085 ts_tablet_manager.cc:1905] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:12.414426 2108 catalog_manager.cc:4841] TS a7fb0ad9c8a34fb09ee3734ecf600956 (127.31.63.66:43571): tablet f731567aa68148738227616c0088b1e2 (table test-table [id=943a126aabea4ae5b7e6e5b434d58abc]) successfully deleted
I20260809 08:42:12.415904 3085 ts_tablet_manager.cc:1918] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:42:12.415972 3085 log.cc:1196] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/f731567aa68148738227616c0088b1e2
I20260809 08:42:12.416304 3085 ts_tablet_manager.cc:1939] T f731567aa68148738227616c0088b1e2 P dc23d7916aec4c19bd1898e1f1144b13: Deleting consensus metadata
I20260809 08:42:12.417191 2106 catalog_manager.cc:4841] TS dc23d7916aec4c19bd1898e1f1144b13 (127.31.63.67:45175): tablet f731567aa68148738227616c0088b1e2 (table test-table [id=943a126aabea4ae5b7e6e5b434d58abc]) successfully deleted
W20260809 08:42:12.423326 2110 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:35985 (error 108)
I20260809 08:42:12.423511 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 2439
I20260809 08:42:12.427639 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:12.427396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35985 (local address 127.0.0.1:58600)
0809 08:42:12.427549 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:12.427603 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:35985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
W20260809 08:42:12.428242 2110 catalog_manager.cc:4584] TS e79bd558378c443cb7139a70215a0c33 (127.31.63.65:35985): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f731567aa68148738227616c0088b1e2: Network error: Client connection negotiation failed: client connection to 127.31.63.65:35985: connect: Connection refused (error 111)
I20260809 08:42:12.434644 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 2656
I20260809 08:42:12.446545 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 2080
2026-08-09T08:42:12Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6150 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:42:12 dist-test-slave-gkw7 krb5kdc[3092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:42:12 dist-test-slave-gkw7 krb5kdc[3092](info): set up 2 sockets
Aug 09 08:42:12 dist-test-slave-gkw7 krb5kdc[3092](info): commencing operation
krb5kdc: starting...
W20260809 08:42:13.683055 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:42:13 dist-test-slave-gkw7 krb5kdc[3092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:42:13.804436 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:39011
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:45771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:39011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:13.886713 3114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:13.886963 3114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:13.887001 3114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:13.888118 3114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:13.888193 3114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:13.888216 3114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:13.888238 3114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:13.888259 3114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:13.889770 3114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:45771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:39011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:39011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:13.890064 3114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:13.890304 3114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:13.895234 3128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:13.895515 3114 server_base.cc:1034] running on GCE node
W20260809 08:42:13.895255 3123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:13.895255 3126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:13.896055 3114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:13.896358 3114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:13.897519 3114 hybrid_clock.cc:648] HybridClock initialized: now 1786264933897505 us; error 43 us; skew 500 ppm
Aug 09 08:42:13 dist-test-slave-gkw7 krb5kdc[3092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264933, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:13.899958 3114 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:42:13.900406 3114 webserver.cc:460] Webserver started at http://127.31.63.126:35601/ using document root <none> and password file <none>
I20260809 08:42:13.900655 3114 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:13.900725 3114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:13.900936 3114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:13.902182 3114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "9ffa6e4e1a454ec28344e9409abaa53a"
format_stamp: "Formatted at 2026-08-09 08:42:13 on dist-test-slave-gkw7"
server_key: "4154119b4e7f6f1f28e153081b2c5af3"
server_key_iv: "d4f88b0bc0046c0b72d020bd0494b2bc"
server_key_version: "encryptionkey@0"
I20260809 08:42:13.902556 3114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "9ffa6e4e1a454ec28344e9409abaa53a"
format_stamp: "Formatted at 2026-08-09 08:42:13 on dist-test-slave-gkw7"
server_key: "4154119b4e7f6f1f28e153081b2c5af3"
server_key_iv: "d4f88b0bc0046c0b72d020bd0494b2bc"
server_key_version: "encryptionkey@0"
I20260809 08:42:13.903992 3114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:42:13.904711 3139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:13.904897 3114 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:13.904999 3114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "9ffa6e4e1a454ec28344e9409abaa53a"
format_stamp: "Formatted at 2026-08-09 08:42:13 on dist-test-slave-gkw7"
server_key: "4154119b4e7f6f1f28e153081b2c5af3"
server_key_iv: "d4f88b0bc0046c0b72d020bd0494b2bc"
server_key_version: "encryptionkey@0"
I20260809 08:42:13.905083 3114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:13.912971 3114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:13.914795 3114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:13.914963 3114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:13.918994 3114 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:39011
I20260809 08:42:13.919065 3216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:39011 every 8 connection(s)
I20260809 08:42:13.919407 3114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:13.919872 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 3114
I20260809 08:42:13.919970 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:13.920181 31997 external_mini_cluster.cc:1428] Setting key 6b7e3bb16455453502cb7922310670d9
I20260809 08:42:13.920506 3218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:13.923336 3218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a: Bootstrap starting.
I20260809 08:42:13.923949 3218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:13.924286 3218 log.cc:824] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:13.925040 3218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a: No bootstrap required, opened a new log
Aug 09 08:42:13 dist-test-slave-gkw7 krb5kdc[3092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:13.927259 3218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6e4e1a454ec28344e9409abaa53a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39011 } }
I20260809 08:42:13.927390 3218 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:13.927412 3218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffa6e4e1a454ec28344e9409abaa53a, State: Initialized, Role: FOLLOWER
I20260809 08:42:13.927575 3218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6e4e1a454ec28344e9409abaa53a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39011 } }
I20260809 08:42:13.927639 3218 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:13.927661 3218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:13.927731 3218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:13.928361 3218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6e4e1a454ec28344e9409abaa53a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39011 } }
I20260809 08:42:13.928449 3218 leader_election.cc:302] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ffa6e4e1a454ec28344e9409abaa53a; no voters:
I20260809 08:42:13.928627 3218 leader_election.cc:288] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:13.928675 3227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:13.928838 3227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [term 1 LEADER]: Becoming Leader. State: Replica: 9ffa6e4e1a454ec28344e9409abaa53a, State: Running, Role: LEADER
I20260809 08:42:13.928938 3218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:13.929018 3227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6e4e1a454ec28344e9409abaa53a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39011 } }
I20260809 08:42:13.929382 3227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ffa6e4e1a454ec28344e9409abaa53a. Latest consensus state: current_term: 1 leader_uuid: "9ffa6e4e1a454ec28344e9409abaa53a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6e4e1a454ec28344e9409abaa53a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39011 } } }
I20260809 08:42:13.929654 3227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:13.929565 3229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ffa6e4e1a454ec28344e9409abaa53a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffa6e4e1a454ec28344e9409abaa53a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39011 } } }
I20260809 08:42:13.930102 3241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:13.930194 3229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:13.930815 3241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:13.931699 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:13.920663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53574 (local address 127.31.63.126:39011)
0809 08:42:13.920843 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:13.920853 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:13.920892 (+ 39us) server_negotiation.cc:407] Connection header received
0809 08:42:13.921634 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:13.921642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:13.921691 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:13.921848 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:13.922486 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:13.923371 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:13.924190 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:13.924773 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:13.924889 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:13.927656 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:13.927660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:13.927661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:13.927673 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:13.929912 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:13.930387 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:13.930390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:13.930391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:13.930508 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:13.930749 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:13.930750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:13.930750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:13.931061 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:13.931219 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:13.931362 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 08:42:13.931501 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260809 08:42:13.932525 3241 catalog_manager.cc:1347] Generated new cluster ID: 7d15470a60fc4eafbeb0c7afa0564286
I20260809 08:42:13.932585 3241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:13.940582 3241 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:13.941073 3241 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:13.946213 3241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ffa6e4e1a454ec28344e9409abaa53a: Generated new TSK 0
I20260809 08:42:13.946379 3241 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:42:13.974454 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:39011
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:45771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:14.058696 3259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:14.058902 3259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:14.058925 3259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:14.060186 3259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:14.060237 3259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:14.060264 3259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:14.061807 3259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:45771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:39011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:14.062067 3259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:14.062314 3259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:14.067503 3269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:14.067517 3271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:14.067495 3268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:14.067636 3259 server_base.cc:1034] running on GCE node
I20260809 08:42:14.067958 3259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:14.068200 3259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:14.069386 3259 hybrid_clock.cc:648] HybridClock initialized: now 1786264934069364 us; error 38 us; skew 500 ppm
Aug 09 08:42:14 dist-test-slave-gkw7 krb5kdc[3092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264934, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:14.071923 3259 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:42:14.072474 3259 webserver.cc:460] Webserver started at http://127.31.63.65:37409/ using document root <none> and password file <none>
I20260809 08:42:14.072724 3259 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:14.072796 3259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:14.072937 3259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:14.074054 3259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "0f5a94ba1d9446469259a3ccc6edb684"
format_stamp: "Formatted at 2026-08-09 08:42:14 on dist-test-slave-gkw7"
server_key: "68188cb9b63c9c69cb0a3cb8b1710a06"
server_key_iv: "1a90919dcca87ea1bc37f7431b6feab3"
server_key_version: "encryptionkey@0"
I20260809 08:42:14.074433 3259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "0f5a94ba1d9446469259a3ccc6edb684"
format_stamp: "Formatted at 2026-08-09 08:42:14 on dist-test-slave-gkw7"
server_key: "68188cb9b63c9c69cb0a3cb8b1710a06"
server_key_iv: "1a90919dcca87ea1bc37f7431b6feab3"
server_key_version: "encryptionkey@0"
I20260809 08:42:14.075784 3259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:14.076493 3282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:14.076668 3259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:14.076781 3259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "0f5a94ba1d9446469259a3ccc6edb684"
format_stamp: "Formatted at 2026-08-09 08:42:14 on dist-test-slave-gkw7"
server_key: "68188cb9b63c9c69cb0a3cb8b1710a06"
server_key_iv: "1a90919dcca87ea1bc37f7431b6feab3"
server_key_version: "encryptionkey@0"
I20260809 08:42:14.076858 3259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:14.093310 3259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:14.095119 3259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:14.095291 3259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:14.095522 3259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:14.095849 3259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:14.095916 3259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:14.095959 3259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:14.095988 3259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:14.102630 3259 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:38323
I20260809 08:42:14.102689 3471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:38323 every 8 connection(s)
I20260809 08:42:14.102996 3259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:42:14 dist-test-slave-gkw7 krb5kdc[3092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264934, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:14.109934 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 3259
I20260809 08:42:14.110013 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:14.110195 31997 external_mini_cluster.cc:1428] Setting key 4232a6939c16b643e12016929b5b202c
I20260809 08:42:14.113037 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:14.103736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:47083 (local address 127.31.63.126:39011)
0809 08:42:14.103782 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:14.103784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:14.104018 (+ 234us) server_negotiation.cc:407] Connection header received
0809 08:42:14.104580 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:14.104582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:14.104596 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:14.104649 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:14.105677 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.106203 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.107242 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.107448 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.107539 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:14.109792 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:14.109793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:14.109794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:14.109802 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:14.111619 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:14.112021 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:14.112023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:14.112023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:14.112090 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:14.112326 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:14.112326 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:14.112327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:14.112481 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:14.112552 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:14.112843 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 08:42:14.112942 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:42:14.113193 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:14.103706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:39011 (local address 127.31.63.65:47083)
0809 08:42:14.103898 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:14.103933 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:14.104455 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:14.104724 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:14.104729 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:14.105323 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:14.105516 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:14.105521 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.106306 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.106307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:14.107152 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:14.107155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.107565 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.107567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:14.107708 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:14.108230 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:14.108240 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:14.109692 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:14.111722 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:14.111729 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:14.111730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:14.111945 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:14.112156 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:14.112157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:14.112157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:14.112260 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:14.112545 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:14.112550 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:14.112660 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 08:42:14.112801 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 08:42:14.112965 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260809 08:42:14.113871 3473 heartbeater.cc:347] Connected to a master server at 127.31.63.126:39011
I20260809 08:42:14.113981 3473 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:14.114161 3473 heartbeater.cc:511] Master 127.31.63.126:39011 requested a full tablet report, sending...
I20260809 08:42:14.114745 3167 ts_manager.cc:194] Registered new tserver with Master: 0f5a94ba1d9446469259a3ccc6edb684 (127.31.63.65:38323)
I20260809 08:42:14.115731 3167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:47083
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:42:14.138460 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:39011
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:45771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:14.221771 3482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:14.221993 3482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:14.222018 3482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:14.223166 3482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:14.223212 3482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:14.223284 3482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:14.224763 3482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:45771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:39011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:14.225018 3482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:14.225258 3482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:14.230419 3490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:14.230449 3492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:14.230420 3496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:14.230844 3482 server_base.cc:1034] running on GCE node
I20260809 08:42:14.231057 3482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:14.231298 3482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:14.232467 3482 hybrid_clock.cc:648] HybridClock initialized: now 1786264934232452 us; error 28 us; skew 500 ppm
Aug 09 08:42:14 dist-test-slave-gkw7 krb5kdc[3092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264934, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:14.239233 3482 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:42:14.239636 3482 webserver.cc:460] Webserver started at http://127.31.63.66:43657/ using document root <none> and password file <none>
I20260809 08:42:14.239890 3482 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:14.239939 3482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:14.240113 3482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:14.241171 3482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "f173ad8c8ed64ef1bcc7fa8a14e5b543"
format_stamp: "Formatted at 2026-08-09 08:42:14 on dist-test-slave-gkw7"
server_key: "7e4b8f6ca142741d86008dd1481edf39"
server_key_iv: "081556cd4abcba88e8081c242412339b"
server_key_version: "encryptionkey@0"
I20260809 08:42:14.241604 3482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "f173ad8c8ed64ef1bcc7fa8a14e5b543"
format_stamp: "Formatted at 2026-08-09 08:42:14 on dist-test-slave-gkw7"
server_key: "7e4b8f6ca142741d86008dd1481edf39"
server_key_iv: "081556cd4abcba88e8081c242412339b"
server_key_version: "encryptionkey@0"
I20260809 08:42:14.243003 3482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:14.243860 3510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:14.244045 3482 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:42:14.244148 3482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "f173ad8c8ed64ef1bcc7fa8a14e5b543"
format_stamp: "Formatted at 2026-08-09 08:42:14 on dist-test-slave-gkw7"
server_key: "7e4b8f6ca142741d86008dd1481edf39"
server_key_iv: "081556cd4abcba88e8081c242412339b"
server_key_version: "encryptionkey@0"
I20260809 08:42:14.244238 3482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:14.276770 3482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:14.278891 3482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:14.279055 3482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:14.279305 3482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:14.279713 3482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:14.279778 3482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:14.279822 3482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:14.279848 3482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:14.286260 3482 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:32781
I20260809 08:42:14.286314 3670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:32781 every 8 connection(s)
I20260809 08:42:14.286634 3482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:42:14 dist-test-slave-gkw7 krb5kdc[3092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264934, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:14.293989 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 3482
I20260809 08:42:14.294092 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:42:14.294324 31997 external_mini_cluster.cc:1428] Setting key 5461a5468b685e37ac2aa7fb6234f513
I20260809 08:42:14.296340 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:14.287367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:53995 (local address 127.31.63.126:39011)
0809 08:42:14.287442 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:14.287445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:14.287662 (+ 217us) server_negotiation.cc:407] Connection header received
0809 08:42:14.288150 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:14.288151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:14.288162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:14.288223 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:14.289179 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.289787 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.290811 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.291003 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.291059 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:14.293241 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:14.293242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:14.293243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:14.293250 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:14.294853 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:14.295370 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:14.295370 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:14.295371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:14.295423 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:14.295709 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:14.295710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:14.295710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:14.295811 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:14.295861 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:14.296203 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 08:42:14.296275 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:42:14.296645 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:14.287344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:39011 (local address 127.31.63.66:53995)
0809 08:42:14.287550 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:14.287588 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:14.288063 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:14.288317 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:14.288320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:14.288901 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:14.289098 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:14.289102 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.289866 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.289867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:14.290722 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:14.290724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.291083 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.291084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:14.291190 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:14.291748 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:14.291757 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:14.293151 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:14.294930 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:14.294937 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:14.294939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:14.295290 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:14.295491 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:14.295492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:14.295492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:14.295640 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:14.295878 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:14.295885 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:14.296060 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 08:42:14.296177 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 08:42:14.296368 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":110,"threads_started":1}
I20260809 08:42:14.297183 3672 heartbeater.cc:347] Connected to a master server at 127.31.63.126:39011
I20260809 08:42:14.297302 3672 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:14.297511 3672 heartbeater.cc:511] Master 127.31.63.126:39011 requested a full tablet report, sending...
I20260809 08:42:14.297930 3167 ts_manager.cc:194] Registered new tserver with Master: f173ad8c8ed64ef1bcc7fa8a14e5b543 (127.31.63.66:32781)
I20260809 08:42:14.298321 3167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:53995
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:42:14.320844 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:39011
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:45771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:14.421295 3683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:14.421492 3683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:14.421515 3683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:14.422725 3683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:14.422771 3683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:14.422797 3683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:14.424242 3683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:45771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:39011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:14.424508 3683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:14.424749 3683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:14.430168 3689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:14.430136 3692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:14.430152 3690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:14.430579 3683 server_base.cc:1034] running on GCE node
I20260809 08:42:14.430790 3683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:14.431041 3683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:14.432209 3683 hybrid_clock.cc:648] HybridClock initialized: now 1786264934432198 us; error 40 us; skew 500 ppm
Aug 09 08:42:14 dist-test-slave-gkw7 krb5kdc[3092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264934, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:14.434760 3683 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:42:14.435215 3683 webserver.cc:460] Webserver started at http://127.31.63.67:40985/ using document root <none> and password file <none>
I20260809 08:42:14.435479 3683 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:14.435551 3683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:14.435704 3683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:14.436877 3683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "e21dbb38e754480c9813549c3df40181"
format_stamp: "Formatted at 2026-08-09 08:42:14 on dist-test-slave-gkw7"
server_key: "4916da0e1db328a0eb0c869f6c933910"
server_key_iv: "ca8d7aca5177a9b003c6515d58b1c599"
server_key_version: "encryptionkey@0"
I20260809 08:42:14.437245 3683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "e21dbb38e754480c9813549c3df40181"
format_stamp: "Formatted at 2026-08-09 08:42:14 on dist-test-slave-gkw7"
server_key: "4916da0e1db328a0eb0c869f6c933910"
server_key_iv: "ca8d7aca5177a9b003c6515d58b1c599"
server_key_version: "encryptionkey@0"
I20260809 08:42:14.438673 3683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:14.439512 3703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:14.439697 3683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:14.439817 3683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "e21dbb38e754480c9813549c3df40181"
format_stamp: "Formatted at 2026-08-09 08:42:14 on dist-test-slave-gkw7"
server_key: "4916da0e1db328a0eb0c869f6c933910"
server_key_iv: "ca8d7aca5177a9b003c6515d58b1c599"
server_key_version: "encryptionkey@0"
I20260809 08:42:14.439924 3683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:14.452016 3683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:14.454061 3683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:14.454239 3683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:14.454495 3683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:14.454838 3683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:14.454898 3683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:14.454974 3683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:14.455008 3683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:14.461853 3683 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:37067
I20260809 08:42:14.461891 3871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:37067 every 8 connection(s)
I20260809 08:42:14.462316 3683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:42:14.466986 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 3683
I20260809 08:42:14.467095 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:14.467315 31997 external_mini_cluster.cc:1428] Setting key 633cf0243799028ac126acb546b9133a
Aug 09 08:42:14 dist-test-slave-gkw7 krb5kdc[3092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264934, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:14.472318 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:14.463170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:54479 (local address 127.31.63.126:39011)
0809 08:42:14.463222 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:14.463224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:14.463459 (+ 235us) server_negotiation.cc:407] Connection header received
0809 08:42:14.464136 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:14.464137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:14.464149 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:14.464210 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:14.465144 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.465754 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.466731 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.467036 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.467116 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:14.469673 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:14.469679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:14.469679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:14.469686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:14.471116 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:14.471523 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:14.471524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:14.471524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:14.471577 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:14.471795 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:14.471796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:14.471796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:14.471897 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:14.471948 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:14.472181 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 08:42:14.472251 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:42:14.472560 3875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:14.463148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:39011 (local address 127.31.63.67:54479)
0809 08:42:14.463336 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:14.463372 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:14.463905 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:14.464290 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:14.464293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:14.464857 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:14.465055 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:14.465060 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.465831 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.465832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:14.466649 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:14.466651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:14.467136 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:14.467137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:14.467277 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:14.467944 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:14.467955 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:14.469446 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:14.471184 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:14.471194 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:14.471196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:14.471457 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:14.471636 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:14.471637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:14.471637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:14.471740 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:14.471953 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:14.471957 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:14.472074 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 08:42:14.472156 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 08:42:14.472294 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260809 08:42:14.473101 3872 heartbeater.cc:347] Connected to a master server at 127.31.63.126:39011
I20260809 08:42:14.473188 3872 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:14.473343 3872 heartbeater.cc:511] Master 127.31.63.126:39011 requested a full tablet report, sending...
I20260809 08:42:14.473728 3167 ts_manager.cc:194] Registered new tserver with Master: e21dbb38e754480c9813549c3df40181 (127.31.63.67:37067)
I20260809 08:42:14.474061 3167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:54479
I20260809 08:42:14.479440 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 08:42:14.493076 3224 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:42:14.492090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53582 (local address 127.31.63.126:39011)
0809 08:42:14.492165 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:14.492167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:14.492322 (+ 155us) server_negotiation.cc:407] Connection header received
0809 08:42:14.492373 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:14.492374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:14.492384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:14.492436 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:14.493005 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53582: BlockingRecv error: recv got EOF from 127.0.0.1:53582 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:42:14.494531 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 3259
I20260809 08:42:14.517673 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 3482
I20260809 08:42:14.530946 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 3683
I20260809 08:42:14.540833 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 3114
2026-08-09T08:42:14Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2096 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:42:14 dist-test-slave-gkw7 krb5kdc[3890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:42:14 dist-test-slave-gkw7 krb5kdc[3890](info): set up 2 sockets
Aug 09 08:42:14 dist-test-slave-gkw7 krb5kdc[3890](info): commencing operation
krb5kdc: starting...
W20260809 08:42:15.764195 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:42:15 dist-test-slave-gkw7 krb5kdc[3890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:42:15.887923 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37703
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:37703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:15.970589 3910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:15.970799 3910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:15.970824 3910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:15.971912 3910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:15.971946 3910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:15.971961 3910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:15.971974 3910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:15.971987 3910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:15.973433 3910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:37703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:15.973706 3910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:15.973904 3910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:15.979261 3916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:15.979312 3910 server_base.cc:1034] running on GCE node
W20260809 08:42:15.979321 3918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:15.979266 3921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:15.979754 3910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:15.980094 3910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:15.981257 3910 hybrid_clock.cc:648] HybridClock initialized: now 1786264935981233 us; error 39 us; skew 500 ppm
Aug 09 08:42:15 dist-test-slave-gkw7 krb5kdc[3890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264935, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:15.988163 3910 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:42:15.988641 3910 webserver.cc:460] Webserver started at http://127.31.63.126:46593/ using document root <none> and password file <none>
I20260809 08:42:15.988915 3910 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:15.988989 3910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:15.989143 3910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:15.990291 3910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "6353ea1d25704436b4d9b50e003aa743"
format_stamp: "Formatted at 2026-08-09 08:42:15 on dist-test-slave-gkw7"
server_key: "9271259b7a27d1868741730d4f0b75c9"
server_key_iv: "1a736fb638a9558b5d93ee2071636836"
server_key_version: "encryptionkey@0"
I20260809 08:42:15.990667 3910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "6353ea1d25704436b4d9b50e003aa743"
format_stamp: "Formatted at 2026-08-09 08:42:15 on dist-test-slave-gkw7"
server_key: "9271259b7a27d1868741730d4f0b75c9"
server_key_iv: "1a736fb638a9558b5d93ee2071636836"
server_key_version: "encryptionkey@0"
I20260809 08:42:15.992080 3910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:42:15.992862 3930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:15.993040 3910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:15.993151 3910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "6353ea1d25704436b4d9b50e003aa743"
format_stamp: "Formatted at 2026-08-09 08:42:15 on dist-test-slave-gkw7"
server_key: "9271259b7a27d1868741730d4f0b75c9"
server_key_iv: "1a736fb638a9558b5d93ee2071636836"
server_key_version: "encryptionkey@0"
I20260809 08:42:15.993233 3910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:16.002554 3910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:16.004987 3910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:16.005160 3910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:16.009711 3910 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:37703
I20260809 08:42:16.009755 4010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:37703 every 8 connection(s)
I20260809 08:42:16.010342 3910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:16.011089 4012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:16.012862 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 3910
I20260809 08:42:16.012972 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:16.013163 31997 external_mini_cluster.cc:1428] Setting key b85b0fb1500dfbacad6b592765215fe3
I20260809 08:42:16.013947 4012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743: Bootstrap starting.
I20260809 08:42:16.014639 4012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:16.014997 4012 log.cc:824] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:16.015761 4012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743: No bootstrap required, opened a new log
I20260809 08:42:16.017725 4012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353ea1d25704436b4d9b50e003aa743" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37703 } }
I20260809 08:42:16.017889 4012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:16.017952 4012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6353ea1d25704436b4d9b50e003aa743, State: Initialized, Role: FOLLOWER
I20260809 08:42:16.018097 4012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353ea1d25704436b4d9b50e003aa743" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37703 } }
I20260809 08:42:16.018191 4012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:16.018222 4012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:16.018265 4012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:16.018870 4012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353ea1d25704436b4d9b50e003aa743" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37703 } }
I20260809 08:42:16.018963 4012 leader_election.cc:302] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6353ea1d25704436b4d9b50e003aa743; no voters:
I20260809 08:42:16.019164 4012 leader_election.cc:288] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:16.019219 4018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:16.019409 4018 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [term 1 LEADER]: Becoming Leader. State: Replica: 6353ea1d25704436b4d9b50e003aa743, State: Running, Role: LEADER
I20260809 08:42:16.019436 4012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:16.019515 4018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353ea1d25704436b4d9b50e003aa743" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37703 } }
I20260809 08:42:16.019879 4018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6353ea1d25704436b4d9b50e003aa743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353ea1d25704436b4d9b50e003aa743" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37703 } } }
I20260809 08:42:16.019958 4018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:16.020082 4018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6353ea1d25704436b4d9b50e003aa743. Latest consensus state: current_term: 1 leader_uuid: "6353ea1d25704436b4d9b50e003aa743" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6353ea1d25704436b4d9b50e003aa743" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37703 } } }
Aug 09 08:42:16 dist-test-slave-gkw7 krb5kdc[3890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:16.020135 4018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:16.020267 4026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:16.020799 4026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:16.024402 4016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:16.013701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.31.63.126:37703)
0809 08:42:16.013905 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:16.013915 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:16.013951 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:42:16.014414 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:16.014421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:16.014471 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:16.014589 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:16.015444 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:16.016443 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:16.017220 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:16.017707 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:16.017847 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:16.020791 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:16.020795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:16.020795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:16.020810 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:16.022782 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:16.023212 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:16.023215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:16.023215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:16.023301 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:16.023514 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:16.023515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:16.023515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:16.023767 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:16.023868 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:16.024022 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 08:42:16.024129 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260809 08:42:16.025447 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:42:16.026816 4026 catalog_manager.cc:1347] Generated new cluster ID: acfd1a25dc3d476e8619907e123c2ba7
I20260809 08:42:16.026865 4026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:16.033411 4026 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:16.034051 4026 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:16.037254 4026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6353ea1d25704436b4d9b50e003aa743: Generated new TSK 0
I20260809 08:42:16.037437 4026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:42:16.043829 4016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:16.040008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33382 (local address 127.31.63.126:37703)
0809 08:42:16.040072 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:16.040075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:16.040172 (+ 97us) server_negotiation.cc:407] Connection header received
0809 08:42:16.040227 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:16.040229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:16.040241 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:16.040324 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:16.040673 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:16.041474 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:16.042271 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:16.042602 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:16.042673 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:16.043357 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:16.043359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:16.043359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:16.043579 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:16.043597 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:16.043604 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:16.043666 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:16.043723 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 08:42:16.043751 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:42:16.046378 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 3910
2026-08-09T08:42:16Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1509 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:42:16 dist-test-slave-gkw7 krb5kdc[4057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:42:16 dist-test-slave-gkw7 krb5kdc[4057](info): set up 2 sockets
Aug 09 08:42:16 dist-test-slave-gkw7 krb5kdc[4057](info): commencing operation
krb5kdc: starting...
W20260809 08:42:17.274362 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:42:17 dist-test-slave-gkw7 krb5kdc[4057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:42:17.404944 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:34975
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:34975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:17.488025 4076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:17.488257 4076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:17.488281 4076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:17.489398 4076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:17.489485 4076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:17.489509 4076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:17.489532 4076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:17.489557 4076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:17.491673 4076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:34975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:34975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:17.492172 4076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:17.492513 4076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:17.498308 4084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:17.498345 4083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:17.498282 4086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:17.498523 4076 server_base.cc:1034] running on GCE node
I20260809 08:42:17.498751 4076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:17.499068 4076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:17.500218 4076 hybrid_clock.cc:648] HybridClock initialized: now 1786264937500200 us; error 29 us; skew 500 ppm
Aug 09 08:42:17 dist-test-slave-gkw7 krb5kdc[4057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264937, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:17.502769 4076 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:42:17.503252 4076 webserver.cc:460] Webserver started at http://127.31.63.126:45099/ using document root <none> and password file <none>
I20260809 08:42:17.503472 4076 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:17.503515 4076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:17.503607 4076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:17.504645 4076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "2a79f90affbd4a769bbb73a1a0b78387"
format_stamp: "Formatted at 2026-08-09 08:42:17 on dist-test-slave-gkw7"
server_key: "fb7abac8f91d9840e47c272edce8cd5b"
server_key_iv: "54402f4f652b6d8017c26147f8920003"
server_key_version: "encryptionkey@0"
I20260809 08:42:17.504972 4076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "2a79f90affbd4a769bbb73a1a0b78387"
format_stamp: "Formatted at 2026-08-09 08:42:17 on dist-test-slave-gkw7"
server_key: "fb7abac8f91d9840e47c272edce8cd5b"
server_key_iv: "54402f4f652b6d8017c26147f8920003"
server_key_version: "encryptionkey@0"
I20260809 08:42:17.506379 4076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:42:17.507097 4098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:17.507289 4076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:17.507403 4076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "2a79f90affbd4a769bbb73a1a0b78387"
format_stamp: "Formatted at 2026-08-09 08:42:17 on dist-test-slave-gkw7"
server_key: "fb7abac8f91d9840e47c272edce8cd5b"
server_key_iv: "54402f4f652b6d8017c26147f8920003"
server_key_version: "encryptionkey@0"
I20260809 08:42:17.507483 4076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:17.518558 4076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:17.520500 4076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:17.520666 4076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:17.525141 4076 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:34975
I20260809 08:42:17.525197 4177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:34975 every 8 connection(s)
I20260809 08:42:17.525648 4076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:17.526839 4179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:17.529146 4179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387: Bootstrap starting.
I20260809 08:42:17.529816 4179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:17.530128 4179 log.cc:824] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:17.530264 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 4076
I20260809 08:42:17.530400 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:17.530614 31997 external_mini_cluster.cc:1428] Setting key d15090e2d337b26ace560d04f6c2e771
I20260809 08:42:17.530915 4179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387: No bootstrap required, opened a new log
I20260809 08:42:17.532701 4179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a79f90affbd4a769bbb73a1a0b78387" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34975 } }
I20260809 08:42:17.532835 4179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:17.532857 4179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a79f90affbd4a769bbb73a1a0b78387, State: Initialized, Role: FOLLOWER
I20260809 08:42:17.533004 4179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a79f90affbd4a769bbb73a1a0b78387" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34975 } }
I20260809 08:42:17.533100 4179 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:17.533144 4179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:17.533207 4179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:17.534144 4179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a79f90affbd4a769bbb73a1a0b78387" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34975 } }
I20260809 08:42:17.534281 4179 leader_election.cc:302] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a79f90affbd4a769bbb73a1a0b78387; no voters:
I20260809 08:42:17.534477 4179 leader_election.cc:288] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:17.534502 4184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:17.534616 4184 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [term 1 LEADER]: Becoming Leader. State: Replica: 2a79f90affbd4a769bbb73a1a0b78387, State: Running, Role: LEADER
I20260809 08:42:17.534751 4179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:17.534766 4184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a79f90affbd4a769bbb73a1a0b78387" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34975 } }
I20260809 08:42:17.535190 4185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a79f90affbd4a769bbb73a1a0b78387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a79f90affbd4a769bbb73a1a0b78387" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34975 } } }
I20260809 08:42:17.535276 4185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:17.535506 4186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a79f90affbd4a769bbb73a1a0b78387. Latest consensus state: current_term: 1 leader_uuid: "2a79f90affbd4a769bbb73a1a0b78387" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a79f90affbd4a769bbb73a1a0b78387" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 34975 } } }
I20260809 08:42:17.535845 4186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387 [sys.catalog]: This master's current role is: LEADER
W20260809 08:42:17.536823 4212 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:42:17.536881 4212 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:42:17.536927 4191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:17.537083 4191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:17.538856 4191 catalog_manager.cc:1347] Generated new cluster ID: 23dc563ffcf74ad297f3395ef15f251a
I20260809 08:42:17.538916 4191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:42:17 dist-test-slave-gkw7 krb5kdc[4057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:17.544068 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:17.531217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.31.63.126:34975)
0809 08:42:17.531444 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:17.531454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:17.531493 (+ 39us) server_negotiation.cc:407] Connection header received
0809 08:42:17.532712 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:17.532722 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:17.532782 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:17.532978 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:17.533899 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:17.535333 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:17.536635 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:17.537152 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:17.537288 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:17.540202 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:17.540206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:17.540206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:17.540219 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:17.542301 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:17.542729 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:17.542733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:17.542733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:17.542853 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:17.543110 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:17.543112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:17.543112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:17.543377 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:17.543591 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:17.543685 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 08:42:17.543841 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
I20260809 08:42:17.545296 4191 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:17.546022 4191 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:17.549670 4191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a79f90affbd4a769bbb73a1a0b78387: Generated new TSK 0
I20260809 08:42:17.549849 4191 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:42:17.580899 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34975
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:17.665930 4219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:17.666230 4219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:17.666258 4219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:17.667445 4219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:17.667488 4219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:17.667515 4219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:17.669018 4219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:17.669282 4219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:17.669557 4219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:17.674849 4225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:17.674855 4226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:17.674856 4230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:17.675567 4219 server_base.cc:1034] running on GCE node
I20260809 08:42:17.675786 4219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:17.676029 4219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:17.677209 4219 hybrid_clock.cc:648] HybridClock initialized: now 1786264937677188 us; error 31 us; skew 500 ppm
Aug 09 08:42:17 dist-test-slave-gkw7 krb5kdc[4057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264937, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:17.679842 4219 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:42:17.680327 4219 webserver.cc:460] Webserver started at http://127.31.63.65:37733/ using document root <none> and password file <none>
I20260809 08:42:17.680523 4219 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:17.680562 4219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:17.680652 4219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:17.681738 4219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "897fc9ac5a834d4c8d958c7baa7acd91"
format_stamp: "Formatted at 2026-08-09 08:42:17 on dist-test-slave-gkw7"
server_key: "f1dab1607c637c86cd614b89708d1544"
server_key_iv: "2f54d94b92bff56dab7eeccf8f938319"
server_key_version: "encryptionkey@0"
I20260809 08:42:17.682085 4219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "897fc9ac5a834d4c8d958c7baa7acd91"
format_stamp: "Formatted at 2026-08-09 08:42:17 on dist-test-slave-gkw7"
server_key: "f1dab1607c637c86cd614b89708d1544"
server_key_iv: "2f54d94b92bff56dab7eeccf8f938319"
server_key_version: "encryptionkey@0"
I20260809 08:42:17.683378 4219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:17.684098 4241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:17.684327 4219 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:17.684413 4219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "897fc9ac5a834d4c8d958c7baa7acd91"
format_stamp: "Formatted at 2026-08-09 08:42:17 on dist-test-slave-gkw7"
server_key: "f1dab1607c637c86cd614b89708d1544"
server_key_iv: "2f54d94b92bff56dab7eeccf8f938319"
server_key_version: "encryptionkey@0"
I20260809 08:42:17.684468 4219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:17.700542 4219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:17.706543 4219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:17.706751 4219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:17.707073 4219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:17.707509 4219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:17.707564 4219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:17.707605 4219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:17.707635 4219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:17.714450 4219 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:36665
I20260809 08:42:17.714563 4400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:36665 every 8 connection(s)
I20260809 08:42:17.714880 4219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:17.716908 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 4219
I20260809 08:42:17.716997 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:17.717159 31997 external_mini_cluster.cc:1428] Setting key dbf09b4a564956ace74b61a35aa73f6e
Aug 09 08:42:17 dist-test-slave-gkw7 krb5kdc[4057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264937, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:17.726554 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:17.715735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:50645 (local address 127.31.63.126:34975)
0809 08:42:17.715826 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:17.715830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:17.716034 (+ 204us) server_negotiation.cc:407] Connection header received
0809 08:42:17.716700 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:17.716701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:17.716745 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:17.716836 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:17.718233 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:17.718803 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:17.720296 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:17.720508 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:17.720562 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:17.723297 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:17.723299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:17.723299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:17.723306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:17.724887 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:17.725433 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:17.725434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:17.725434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:17.725491 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:17.725866 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:17.725867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:17.725867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:17.725984 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:17.726039 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:17.726414 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 08:42:17.726487 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:42:17.726899 4405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:17.715708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:34975 (local address 127.31.63.65:50645)
0809 08:42:17.715901 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:17.715941 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:17.716532 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:17.716929 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:17.716933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:17.717890 (+ 957us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:17.718130 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:17.718134 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:17.718920 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:17.718921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:17.720184 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:17.720188 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:17.720606 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:17.720607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:17.720779 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:17.721444 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:17.721453 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:17.723197 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:17.724985 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:17.724995 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:17.724997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:17.725343 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:17.725621 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:17.725623 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:17.725623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:17.725786 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:17.726063 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:17.726071 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:17.726262 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 08:42:17.726395 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 08:42:17.726612 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
I20260809 08:42:17.727506 4401 heartbeater.cc:347] Connected to a master server at 127.31.63.126:34975
I20260809 08:42:17.727595 4401 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:17.727789 4401 heartbeater.cc:511] Master 127.31.63.126:34975 requested a full tablet report, sending...
I20260809 08:42:17.728368 4123 ts_manager.cc:194] Registered new tserver with Master: 897fc9ac5a834d4c8d958c7baa7acd91 (127.31.63.65:36665)
I20260809 08:42:17.729045 4123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:50645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:42:17.748708 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34975
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:17.833693 4414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:17.833963 4414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:17.834009 4414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:17.835320 4414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:17.835390 4414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:17.835436 4414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:17.837102 4414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:17.837387 4414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:17.837662 4414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:17.842840 4421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:17.842836 4424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:17.842861 4422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:17.843109 4414 server_base.cc:1034] running on GCE node
I20260809 08:42:17.843343 4414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:17.843590 4414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:17.844772 4414 hybrid_clock.cc:648] HybridClock initialized: now 1786264937844758 us; error 27 us; skew 500 ppm
Aug 09 08:42:17 dist-test-slave-gkw7 krb5kdc[4057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264937, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:17.847327 4414 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:42:17.847816 4414 webserver.cc:460] Webserver started at http://127.31.63.66:37005/ using document root <none> and password file <none>
I20260809 08:42:17.848073 4414 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:17.848147 4414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:17.848307 4414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:17.849370 4414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "83a9f531cd8043918d6d994fd3093484"
format_stamp: "Formatted at 2026-08-09 08:42:17 on dist-test-slave-gkw7"
server_key: "12ef73bb344f938b0e68ddbfca5d70e1"
server_key_iv: "6723a1ef58182c2f743d58e271f0091d"
server_key_version: "encryptionkey@0"
I20260809 08:42:17.849793 4414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "83a9f531cd8043918d6d994fd3093484"
format_stamp: "Formatted at 2026-08-09 08:42:17 on dist-test-slave-gkw7"
server_key: "12ef73bb344f938b0e68ddbfca5d70e1"
server_key_iv: "6723a1ef58182c2f743d58e271f0091d"
server_key_version: "encryptionkey@0"
I20260809 08:42:17.851289 4414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:17.852177 4432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:17.852399 4414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:17.852512 4414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "83a9f531cd8043918d6d994fd3093484"
format_stamp: "Formatted at 2026-08-09 08:42:17 on dist-test-slave-gkw7"
server_key: "12ef73bb344f938b0e68ddbfca5d70e1"
server_key_iv: "6723a1ef58182c2f743d58e271f0091d"
server_key_version: "encryptionkey@0"
I20260809 08:42:17.852591 4414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:17.879426 4414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:17.881480 4414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:17.881686 4414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:17.881978 4414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:17.882340 4414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:17.882398 4414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:17.882445 4414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:17.882469 4414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:17.889361 4414 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:37155
I20260809 08:42:17.889424 4617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:37155 every 8 connection(s)
I20260809 08:42:17.889819 4414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:42:17.894829 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 4414
I20260809 08:42:17.894968 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:42:17.895182 31997 external_mini_cluster.cc:1428] Setting key 38c559911e65b9a12442f795e0775acb
Aug 09 08:42:17 dist-test-slave-gkw7 krb5kdc[4057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264937, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:17.900045 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:17.890679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:48761 (local address 127.31.63.126:34975)
0809 08:42:17.890785 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:17.890788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:17.890973 (+ 185us) server_negotiation.cc:407] Connection header received
0809 08:42:17.891472 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:17.891474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:17.891484 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:17.891534 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:17.892506 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:17.893010 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:17.894131 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:17.894334 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:17.894391 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:17.896929 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:17.896931 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:17.896931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:17.896940 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:17.898539 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:17.898992 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:17.898994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:17.898994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:17.899060 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:17.899343 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:17.899344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:17.899345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:17.899473 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:17.899551 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:17.899852 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 08:42:17.899950 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 08:42:17.900282 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:17.890637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:34975 (local address 127.31.63.66:48761)
0809 08:42:17.890846 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:17.890885 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:17.891388 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:17.891597 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:17.891600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:17.892213 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:17.892427 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:17.892432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:17.893093 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:17.893094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:17.894051 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:17.894053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:17.894468 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:17.894470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:17.894634 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:17.895333 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:17.895345 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:17.896821 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:17.898651 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:17.898659 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:17.898662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:17.898908 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:17.899129 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:17.899130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:17.899130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:17.899264 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:17.899544 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:17.899553 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:17.899706 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 08:42:17.899819 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 08:42:17.899997 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":123,"threads_started":1}
I20260809 08:42:17.900725 4619 heartbeater.cc:347] Connected to a master server at 127.31.63.126:34975
I20260809 08:42:17.900840 4619 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:17.901043 4619 heartbeater.cc:511] Master 127.31.63.126:34975 requested a full tablet report, sending...
I20260809 08:42:17.901520 4123 ts_manager.cc:194] Registered new tserver with Master: 83a9f531cd8043918d6d994fd3093484 (127.31.63.66:37155)
I20260809 08:42:17.901948 4123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:48761
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:42:17.924964 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34975
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:18.012394 4629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:18.012616 4629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:18.012642 4629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:18.013924 4629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:18.013978 4629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:18.014005 4629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:18.015570 4629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:34975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:18.015864 4629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:18.016125 4629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:18.021716 4638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:18.021703 4637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:18.021698 4641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:18.022475 4629 server_base.cc:1034] running on GCE node
I20260809 08:42:18.022704 4629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:18.022939 4629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:18.024092 4629 hybrid_clock.cc:648] HybridClock initialized: now 1786264938024072 us; error 43 us; skew 500 ppm
Aug 09 08:42:18 dist-test-slave-gkw7 krb5kdc[4057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264938, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:18.027019 4629 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:42:18.027552 4629 webserver.cc:460] Webserver started at http://127.31.63.67:45571/ using document root <none> and password file <none>
I20260809 08:42:18.027812 4629 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:18.027884 4629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:18.028043 4629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:18.029183 4629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "9ab6fb4c295f4353b6e4e23c7a572393"
format_stamp: "Formatted at 2026-08-09 08:42:18 on dist-test-slave-gkw7"
server_key: "f7430b2b49b431f507d69ffa9d8ca162"
server_key_iv: "397259594baf0d9f7162db1e9853fb95"
server_key_version: "encryptionkey@0"
I20260809 08:42:18.029644 4629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "9ab6fb4c295f4353b6e4e23c7a572393"
format_stamp: "Formatted at 2026-08-09 08:42:18 on dist-test-slave-gkw7"
server_key: "f7430b2b49b431f507d69ffa9d8ca162"
server_key_iv: "397259594baf0d9f7162db1e9853fb95"
server_key_version: "encryptionkey@0"
I20260809 08:42:18.033406 4629 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260809 08:42:18.037078 4653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:18.037317 4629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 08:42:18.037438 4629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "9ab6fb4c295f4353b6e4e23c7a572393"
format_stamp: "Formatted at 2026-08-09 08:42:18 on dist-test-slave-gkw7"
server_key: "f7430b2b49b431f507d69ffa9d8ca162"
server_key_iv: "397259594baf0d9f7162db1e9853fb95"
server_key_version: "encryptionkey@0"
I20260809 08:42:18.037602 4629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:18.048803 4629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:18.051411 4629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:18.051541 4629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:18.051864 4629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:18.052222 4629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:18.052261 4629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:18.052285 4629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:18.052299 4629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:18.059108 4629 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:36947
I20260809 08:42:18.059159 4814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:36947 every 8 connection(s)
I20260809 08:42:18.059521 4629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:42:18.065132 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 4629
I20260809 08:42:18.065274 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:18.065467 31997 external_mini_cluster.cc:1428] Setting key dd692101639e1bdf2dfcb5d0b7a68b48
Aug 09 08:42:18 dist-test-slave-gkw7 krb5kdc[4057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264938, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:18.070118 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:18.060330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:50737 (local address 127.31.63.126:34975)
0809 08:42:18.060403 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:18.060405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:18.060645 (+ 240us) server_negotiation.cc:407] Connection header received
0809 08:42:18.061164 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:18.061165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:18.061176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:18.061246 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:18.062375 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:18.062916 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:18.064042 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:18.064264 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:18.064327 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:18.067043 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:18.067045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:18.067045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:18.067053 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:18.068623 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:18.069033 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:18.069034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:18.069034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:18.069091 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:18.069346 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:18.069347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:18.069347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:18.069496 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:18.069654 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:18.069964 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 08:42:18.070049 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 08:42:18.070286 4818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:18.060331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:34975 (local address 127.31.63.67:50737)
0809 08:42:18.060510 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:18.060548 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:18.061076 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:18.061315 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:18.061318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:18.062038 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:18.062257 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:18.062263 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:18.063044 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:18.063046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:18.063941 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:18.063944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:18.064353 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:18.064354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:18.064467 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:18.065076 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:18.065085 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:18.066957 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:18.068717 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:18.068726 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:18.068728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:18.068964 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:18.069150 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:18.069150 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:18.069151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:18.069263 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:18.069666 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:18.069672 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:18.069814 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 08:42:18.069909 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 08:42:18.070051 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260809 08:42:18.070765 4815 heartbeater.cc:347] Connected to a master server at 127.31.63.126:34975
I20260809 08:42:18.070892 4815 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:18.071112 4815 heartbeater.cc:511] Master 127.31.63.126:34975 requested a full tablet report, sending...
I20260809 08:42:18.071581 4123 ts_manager.cc:194] Registered new tserver with Master: 9ab6fb4c295f4353b6e4e23c7a572393 (127.31.63.67:36947)
I20260809 08:42:18.071991 4123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:50737
I20260809 08:42:18.077451 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:42:18 dist-test-slave-gkw7 krb5kdc[4057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264938, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 08:42:18 dist-test-slave-gkw7 krb5kdc[4057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264938, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:18.102434 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:18.094865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52826 (local address 127.31.63.126:34975)
0809 08:42:18.094938 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:18.094941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:18.095093 (+ 152us) server_negotiation.cc:407] Connection header received
0809 08:42:18.095141 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:18.095142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:18.095152 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:18.095205 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:18.096077 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:18.096728 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:18.097459 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:18.097696 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:18.097760 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:18.099925 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:18.099926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:18.099927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:18.099934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:18.101384 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:18.101734 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:18.101734 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:18.101735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:18.101785 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:18.101943 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:18.101943 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:18.101944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:18.102081 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:18.102154 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:18.102246 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 08:42:18.102311 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 08:42:18.102739 4123 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52826
I20260809 08:42:18.103183 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 4219
I20260809 08:42:18.128015 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 4414
I20260809 08:42:18.144573 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 4629
I20260809 08:42:18.153977 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 4076
2026-08-09T08:42:18Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2106 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:42:18 dist-test-slave-gkw7 krb5kdc[4833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:42:18 dist-test-slave-gkw7 krb5kdc[4833](info): set up 2 sockets
Aug 09 08:42:18 dist-test-slave-gkw7 krb5kdc[4833](info): commencing operation
krb5kdc: starting...
W20260809 08:42:19.500579 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 08:42:19.623699 31997 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.123s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 08:42:19.757532 31997 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.134s user 0.002s sys 0.000s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 08:42:19.990505 31997 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.233s user 0.002s sys 0.000s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/test-user.keytab.
W20260809 08:42:20.268033 31997 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.277s user 0.002s sys 0.000s
Aug 09 08:42:20 dist-test-slave-gkw7 krb5kdc[4833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:42:20.317441 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:45697
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:40589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:45697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:20.404219 4858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:20.404433 4858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:20.404455 4858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:20.405635 4858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:20.405675 4858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:20.405689 4858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:20.405730 4858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:20.405759 4858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:20.407382 4858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:45697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:45697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:20.407629 4858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:20.407855 4858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:20.412899 4864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:20.412904 4869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:20.412953 4858 server_base.cc:1034] running on GCE node
W20260809 08:42:20.412899 4863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:20.413323 4858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:20.413699 4858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:20.414847 4858 hybrid_clock.cc:648] HybridClock initialized: now 1786264940414825 us; error 37 us; skew 500 ppm
Aug 09 08:42:20 dist-test-slave-gkw7 krb5kdc[4833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:20.417330 4858 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:42:20.417819 4858 webserver.cc:460] Webserver started at http://127.31.63.126:42779/ using document root <none> and password file <none>
I20260809 08:42:20.418092 4858 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:20.418139 4858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:20.418255 4858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:20.419272 4858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "695b2cc2e1cf423d90a9643f99e45005"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "2d8d4c19c02e96e406f036a997661e2b"
server_key_iv: "0aee0a3e7d418647ff7505a54edbbc1a"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.419579 4858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "695b2cc2e1cf423d90a9643f99e45005"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "2d8d4c19c02e96e406f036a997661e2b"
server_key_iv: "0aee0a3e7d418647ff7505a54edbbc1a"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.420872 4858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:42:20.421713 4878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:20.421900 4858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:42:20.421993 4858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "695b2cc2e1cf423d90a9643f99e45005"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "2d8d4c19c02e96e406f036a997661e2b"
server_key_iv: "0aee0a3e7d418647ff7505a54edbbc1a"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.422051 4858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:20.432833 4858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:20.434864 4858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:20.434980 4858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:20.439097 4858 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:45697
I20260809 08:42:20.439150 4950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:45697 every 8 connection(s)
I20260809 08:42:20.439651 4858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:20.440385 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:20.442812 4951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005: Bootstrap starting.
I20260809 08:42:20.443320 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 4858
I20260809 08:42:20.443409 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:20.443506 4951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:20.443641 31997 external_mini_cluster.cc:1428] Setting key 07a76633ea04bcce2cda1c83bd4c3401
I20260809 08:42:20.443897 4951 log.cc:824] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:20.444720 4951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005: No bootstrap required, opened a new log
I20260809 08:42:20.447068 4951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "695b2cc2e1cf423d90a9643f99e45005" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45697 } }
I20260809 08:42:20.447232 4951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:20.447299 4951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 695b2cc2e1cf423d90a9643f99e45005, State: Initialized, Role: FOLLOWER
I20260809 08:42:20.447450 4951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "695b2cc2e1cf423d90a9643f99e45005" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45697 } }
I20260809 08:42:20.447532 4951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:20.447575 4951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:20.447623 4951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:20.448150 4951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "695b2cc2e1cf423d90a9643f99e45005" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45697 } }
I20260809 08:42:20.448269 4951 leader_election.cc:302] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 695b2cc2e1cf423d90a9643f99e45005; no voters:
I20260809 08:42:20.448455 4951 leader_election.cc:288] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:20.448513 4956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:20.448664 4956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [term 1 LEADER]: Becoming Leader. State: Replica: 695b2cc2e1cf423d90a9643f99e45005, State: Running, Role: LEADER
I20260809 08:42:20.448745 4951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:20.448767 4956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "695b2cc2e1cf423d90a9643f99e45005" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45697 } }
I20260809 08:42:20.449468 4957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "695b2cc2e1cf423d90a9643f99e45005" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "695b2cc2e1cf423d90a9643f99e45005" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45697 } } }
I20260809 08:42:20.449595 4957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:20.449857 4958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 695b2cc2e1cf423d90a9643f99e45005. Latest consensus state: current_term: 1 leader_uuid: "695b2cc2e1cf423d90a9643f99e45005" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "695b2cc2e1cf423d90a9643f99e45005" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45697 } } }
I20260809 08:42:20.449997 4958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:20.450333 4968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 08:42:20 dist-test-slave-gkw7 krb5kdc[4833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:20.451113 4968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:20.452852 4968 catalog_manager.cc:1347] Generated new cluster ID: dcddcde3241c4918b22d7e13be5eeb63
I20260809 08:42:20.452913 4968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:20.455140 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:20.444203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52776 (local address 127.31.63.126:45697)
0809 08:42:20.444407 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:20.444418 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:20.444455 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:42:20.444976 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:20.444986 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:20.445054 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:20.445213 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:20.446210 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.447096 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.448049 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.448635 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.448788 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:20.451595 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:20.451599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:20.451599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:20.451611 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:20.453614 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:20.453985 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:20.453988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:20.453988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:20.454085 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:20.454302 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:20.454303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:20.454303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:20.454539 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:20.454719 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:20.454827 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 08:42:20.454968 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":96,"threads_started":1}
I20260809 08:42:20.475075 4968 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:20.475672 4968 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:20.479074 4968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 695b2cc2e1cf423d90a9643f99e45005: Generated new TSK 0
I20260809 08:42:20.479291 4968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:42:20.516381 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:45697
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:40589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:20.601574 4984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:20.601840 4984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:20.601881 4984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:20.603072 4984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:20.603142 4984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:20.603195 4984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:20.604739 4984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:45697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:20.605005 4984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:20.605250 4984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:20.610291 4992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:20.610484 4991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:20.610498 4996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:20.610996 4984 server_base.cc:1034] running on GCE node
I20260809 08:42:20.611266 4984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:20.611474 4984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:20.612627 4984 hybrid_clock.cc:648] HybridClock initialized: now 1786264940612614 us; error 32 us; skew 500 ppm
Aug 09 08:42:20 dist-test-slave-gkw7 krb5kdc[4833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:20.615280 4984 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:42:20.615767 4984 webserver.cc:460] Webserver started at http://127.31.63.65:43461/ using document root <none> and password file <none>
I20260809 08:42:20.615983 4984 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:20.616024 4984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:20.616117 4984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:20.617143 4984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "60d379a8f739491da0c12d380b7aeaf9"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "e2e9060ee5a8d55578f7d3af2a5ec1f3"
server_key_iv: "11d010b10007e2889755adc0b37c01f8"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.617451 4984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "60d379a8f739491da0c12d380b7aeaf9"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "e2e9060ee5a8d55578f7d3af2a5ec1f3"
server_key_iv: "11d010b10007e2889755adc0b37c01f8"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.618949 4984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:20.619634 5006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:20.619818 4984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:42:20.619911 4984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "60d379a8f739491da0c12d380b7aeaf9"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "e2e9060ee5a8d55578f7d3af2a5ec1f3"
server_key_iv: "11d010b10007e2889755adc0b37c01f8"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.619985 4984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:20.650760 4984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:20.652753 4984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:20.652949 4984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:20.653220 4984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:20.653630 4984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:20.653684 4984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:20.653719 4984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:20.653749 4984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:20.660007 4984 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:33969
I20260809 08:42:20.660059 5167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:33969 every 8 connection(s)
I20260809 08:42:20.660404 4984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:20.662787 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 4984
I20260809 08:42:20.662899 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:20.663101 31997 external_mini_cluster.cc:1428] Setting key c8c32c24cf82ff7f52ddf9850074ebd9
Aug 09 08:42:20 dist-test-slave-gkw7 krb5kdc[4833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:20.672124 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:20.661213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:46779 (local address 127.31.63.126:45697)
0809 08:42:20.661278 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:20.661280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:20.661453 (+ 173us) server_negotiation.cc:407] Connection header received
0809 08:42:20.662463 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:20.662464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:20.662475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:20.662539 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:20.663527 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.664091 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.665672 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.666033 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.666110 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:20.669149 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:20.669150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:20.669150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:20.669157 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:20.670668 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:20.671083 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:20.671084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:20.671085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:20.671153 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:20.671425 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:20.671427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:20.671427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:20.671550 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:20.671624 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:20.671923 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 08:42:20.672031 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:42:20.672295 5170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:20.661134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:45697 (local address 127.31.63.65:46779)
0809 08:42:20.661319 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:20.661364 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:20.662361 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:20.662677 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:20.662682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:20.663088 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:20.663422 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:20.663429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.664190 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.664191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:20.665565 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:20.665569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.666126 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.666128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:20.666242 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:20.667054 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:20.667063 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:20.669037 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:20.670756 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:20.670764 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:20.670767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:20.670999 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:20.671228 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:20.671229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:20.671229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:20.671346 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:20.671626 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:20.671634 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:20.671782 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 08:42:20.671887 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 08:42:20.672054 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260809 08:42:20.672813 5168 heartbeater.cc:347] Connected to a master server at 127.31.63.126:45697
I20260809 08:42:20.672907 5168 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
I20260809 08:42:20.673111 5168 heartbeater.cc:511] Master 127.31.63.126:45697 requested a full tablet report, sending...
I20260809 08:42:20.673725 4904 ts_manager.cc:194] Registered new tserver with Master: 60d379a8f739491da0c12d380b7aeaf9 (127.31.63.65:33969)
I20260809 08:42:20.674293 4904 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:46779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:42:20.690761 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:45697
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:40589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:20.776116 5173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:20.776335 5173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:20.776360 5173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:20.777463 5173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:20.777498 5173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:20.777547 5173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:20.779011 5173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:45697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:20.779209 5173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:20.779393 5173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:42:20.784646 5173 server_base.cc:1034] running on GCE node
W20260809 08:42:20.784705 5182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:20.784629 5179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:20.784631 5185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:20.785055 5173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:20.785282 5173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:20.786425 5173 hybrid_clock.cc:648] HybridClock initialized: now 1786264940786412 us; error 32 us; skew 500 ppm
Aug 09 08:42:20 dist-test-slave-gkw7 krb5kdc[4833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:20.788930 5173 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:42:20.789335 5173 webserver.cc:460] Webserver started at http://127.31.63.66:41977/ using document root <none> and password file <none>
I20260809 08:42:20.789592 5173 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:20.789644 5173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:20.789806 5173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:20.790797 5173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "2e3fe1c9a52c4e37b51ee17d12430b53"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "cce3fa6eef17663c17918ff812f1ec19"
server_key_iv: "cb38940a84c59094080bdbce5096528c"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.791090 5173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "2e3fe1c9a52c4e37b51ee17d12430b53"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "cce3fa6eef17663c17918ff812f1ec19"
server_key_iv: "cb38940a84c59094080bdbce5096528c"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.792369 5173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:20.793048 5195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:20.793223 5173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:20.793334 5173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "2e3fe1c9a52c4e37b51ee17d12430b53"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "cce3fa6eef17663c17918ff812f1ec19"
server_key_iv: "cb38940a84c59094080bdbce5096528c"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.793409 5173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:20.802215 5173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:20.804103 5173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:20.804270 5173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:20.804503 5173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:20.804821 5173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:20.804873 5173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:20.804914 5173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:20.804943 5173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:20.811327 5173 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:34631
I20260809 08:42:20.811393 5361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:34631 every 8 connection(s)
I20260809 08:42:20.811703 5173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:42:20.816555 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 5173
I20260809 08:42:20.816692 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:42:20.816897 31997 external_mini_cluster.cc:1428] Setting key e6c9d044c53d4c163dbba5d238dbc633
Aug 09 08:42:20 dist-test-slave-gkw7 krb5kdc[4833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:20.821870 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:20.812489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:42477 (local address 127.31.63.126:45697)
0809 08:42:20.812568 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:20.812570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:20.812733 (+ 163us) server_negotiation.cc:407] Connection header received
0809 08:42:20.813772 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:20.813773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:20.813784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:20.813850 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:20.814446 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.814946 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.815936 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.816169 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.816226 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:20.818769 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:20.818770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:20.818770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:20.818778 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:20.820402 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:20.820815 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:20.820817 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:20.820818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:20.820883 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:20.821103 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:20.821104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:20.821104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:20.821259 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:20.821333 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:20.821608 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 08:42:20.821714 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:42:20.821970 5364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:20.812395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:45697 (local address 127.31.63.66:42477)
0809 08:42:20.812591 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:20.812634 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:20.813674 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:20.813909 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:20.813912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:20.814169 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:20.814376 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:20.814381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.815031 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.815032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:20.815865 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:20.815867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.816264 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.816265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:20.816379 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:20.816997 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:20.817008 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:20.818651 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:20.820486 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:20.820493 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:20.820495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:20.820737 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:20.820949 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:20.820950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:20.820950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:20.821052 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:20.821326 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:20.821332 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:20.821449 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 08:42:20.821593 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 08:42:20.821751 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":122,"threads_started":1}
I20260809 08:42:20.822486 5362 heartbeater.cc:347] Connected to a master server at 127.31.63.126:45697
I20260809 08:42:20.822607 5362 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:20.822832 5362 heartbeater.cc:511] Master 127.31.63.126:45697 requested a full tablet report, sending...
I20260809 08:42:20.823307 4904 ts_manager.cc:194] Registered new tserver with Master: 2e3fe1c9a52c4e37b51ee17d12430b53 (127.31.63.66:34631)
I20260809 08:42:20.823881 4904 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:42477
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:42:20.844367 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:45697
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:40589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:20.929765 5368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:20.929983 5368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:20.930007 5368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:20.931176 5368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:20.931227 5368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:20.931255 5368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:20.932785 5368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:45697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:20.933048 5368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:20.933295 5368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:20.938511 5376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:20.938511 5379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:20.938696 5368 server_base.cc:1034] running on GCE node
W20260809 08:42:20.938601 5377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:20.938977 5368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:20.939234 5368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:20.940425 5368 hybrid_clock.cc:648] HybridClock initialized: now 1786264940940399 us; error 41 us; skew 500 ppm
Aug 09 08:42:20 dist-test-slave-gkw7 krb5kdc[4833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:20.943094 5368 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:42:20.943585 5368 webserver.cc:460] Webserver started at http://127.31.63.67:44419/ using document root <none> and password file <none>
I20260809 08:42:20.943837 5368 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:20.943908 5368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:20.944063 5368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:20.945143 5368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "779582385e54464ebfda3a48a824cef2"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "68ef62141ab5e2a1ec00cf72dc2bc6a7"
server_key_iv: "863a83c9d93de589c9569277ca6f9ff8"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.945515 5368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "779582385e54464ebfda3a48a824cef2"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "68ef62141ab5e2a1ec00cf72dc2bc6a7"
server_key_iv: "863a83c9d93de589c9569277ca6f9ff8"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.946914 5368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:20.947701 5392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:20.947908 5368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:42:20.948019 5368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "779582385e54464ebfda3a48a824cef2"
format_stamp: "Formatted at 2026-08-09 08:42:20 on dist-test-slave-gkw7"
server_key: "68ef62141ab5e2a1ec00cf72dc2bc6a7"
server_key_iv: "863a83c9d93de589c9569277ca6f9ff8"
server_key_version: "encryptionkey@0"
I20260809 08:42:20.948104 5368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:20.964073 5368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:20.966014 5368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:20.966176 5368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:20.966429 5368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:20.966800 5368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:20.966857 5368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:20.966900 5368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:20.966928 5368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:20.973340 5368 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:45813
I20260809 08:42:20.973392 5568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:45813 every 8 connection(s)
I20260809 08:42:20.973752 5368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:42:20 dist-test-slave-gkw7 krb5kdc[4833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264940, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:20.980278 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 5368
I20260809 08:42:20.980398 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:20.980609 31997 external_mini_cluster.cc:1428] Setting key 42c5483e309fc88bc62ae558f601ec8d
I20260809 08:42:20.983506 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:20.974495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:39059 (local address 127.31.63.126:45697)
0809 08:42:20.974546 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:20.974549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:20.974782 (+ 233us) server_negotiation.cc:407] Connection header received
0809 08:42:20.975766 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:20.975767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:20.975779 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:20.975834 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:20.976492 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.976996 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.978059 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.978286 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.978356 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:20.980926 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:20.980928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:20.980929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:20.980937 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:20.982330 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:20.982732 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:20.982733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:20.982734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:20.982783 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:20.982971 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:20.982971 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:20.982972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:20.983075 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:20.983125 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:20.983375 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 08:42:20.983440 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:42:20.983726 5572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:20.974454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:45697 (local address 127.31.63.67:39059)
0809 08:42:20.974629 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:20.974690 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:20.975664 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:20.975926 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:20.975929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:20.976200 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:20.976414 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:20.976419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.977079 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.977080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:20.977950 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:20.977953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:20.978411 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:20.978413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:20.978544 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:20.979212 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:20.979221 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:20.980803 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:20.982409 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:20.982417 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:20.982419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:20.982662 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:20.982824 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:20.982824 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:20.982825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:20.982920 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:20.983135 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:20.983141 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:20.983264 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 08:42:20.983350 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:42:20.983477 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260809 08:42:20.984309 5569 heartbeater.cc:347] Connected to a master server at 127.31.63.126:45697
I20260809 08:42:20.984424 5569 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:20.984620 5569 heartbeater.cc:511] Master 127.31.63.126:45697 requested a full tablet report, sending...
I20260809 08:42:20.985097 4904 ts_manager.cc:194] Registered new tserver with Master: 779582385e54464ebfda3a48a824cef2 (127.31.63.67:45813)
I20260809 08:42:20.985488 4904 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:39059
I20260809 08:42:20.992669 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:42:21 dist-test-slave-gkw7 krb5kdc[4833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:42:21 dist-test-slave-gkw7 krb5kdc[4833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:42:21.022758 5578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:21.009854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.31.63.65:33969)
0809 08:42:21.010036 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:21.010042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:21.010062 (+ 20us) server_negotiation.cc:407] Connection header received
0809 08:42:21.010092 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:21.010094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:21.010109 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:21.010179 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:21.010806 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:21.011548 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:21.012396 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:21.012742 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:21.012813 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:21.019695 (+ 6882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:21.019706 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:21.019706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:21.019715 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:21.021499 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:21.021902 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:21.021905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:21.021905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:21.021969 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:21.022136 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:21.022136 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:21.022137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:21.022301 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:21.022436 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:21.022472 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 08:42:21.022588 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
Aug 09 08:42:21 dist-test-slave-gkw7 krb5kdc[4833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:21.031502 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:21.024168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52788 (local address 127.31.63.126:45697)
0809 08:42:21.024235 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:21.024237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:21.024245 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:21.024268 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:21.024269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:21.024278 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:21.024323 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:21.024784 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:21.025326 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:21.026102 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:21.026302 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:21.026356 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:21.028925 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:21.028927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:21.028928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:21.028935 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:21.030388 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:21.030747 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:21.030748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:21.030748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:21.030799 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:21.030975 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:21.030976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:21.030976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:21.031119 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:21.031213 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:21.031354 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 08:42:21.031428 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 08:42:21.031791 4904 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52788
I20260809 08:42:21.032238 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 4984
I20260809 08:42:21.042727 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 5173
I20260809 08:42:21.055461 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 5368
I20260809 08:42:21.074178 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 4858
2026-08-09T08:42:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2916 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:42:21 dist-test-slave-gkw7 krb5kdc[5585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:42:21 dist-test-slave-gkw7 krb5kdc[5585](info): set up 2 sockets
Aug 09 08:42:21 dist-test-slave-gkw7 krb5kdc[5585](info): commencing operation
krb5kdc: starting...
W20260809 08:42:22.327381 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:42:22 dist-test-slave-gkw7 krb5kdc[5585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:42:22.453926 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33271
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:22.538189 5604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:22.538431 5604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:22.538470 5604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:22.539626 5604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:22.539693 5604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:22.539721 5604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:22.539750 5604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:22.539783 5604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:22.541286 5604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:22.541622 5604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:22.541877 5604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:22.546880 5610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:22.546885 5613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:22.546898 5611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:22.547224 5604 server_base.cc:1034] running on GCE node
I20260809 08:42:22.547514 5604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:22.547809 5604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:22.548998 5604 hybrid_clock.cc:648] HybridClock initialized: now 1786264942548976 us; error 30 us; skew 500 ppm
Aug 09 08:42:22 dist-test-slave-gkw7 krb5kdc[5585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264942, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:22.551683 5604 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:42:22.552103 5604 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:22.552179 5604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:22.552328 5604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:22.553407 5604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "7b2c026d11ed42f4a33ca6e3adca3f00"
format_stamp: "Formatted at 2026-08-09 08:42:22 on dist-test-slave-gkw7"
server_key: "cd554d7aa54e34ce9f86d7fac69a8a1c"
server_key_iv: "a8159778fc82b68224974a5badc20aef"
server_key_version: "encryptionkey@0"
I20260809 08:42:22.553851 5604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "7b2c026d11ed42f4a33ca6e3adca3f00"
format_stamp: "Formatted at 2026-08-09 08:42:22 on dist-test-slave-gkw7"
server_key: "cd554d7aa54e34ce9f86d7fac69a8a1c"
server_key_iv: "a8159778fc82b68224974a5badc20aef"
server_key_version: "encryptionkey@0"
I20260809 08:42:22.555182 5604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:22.555965 5623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:22.556166 5604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:22.556278 5604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "7b2c026d11ed42f4a33ca6e3adca3f00"
format_stamp: "Formatted at 2026-08-09 08:42:22 on dist-test-slave-gkw7"
server_key: "cd554d7aa54e34ce9f86d7fac69a8a1c"
server_key_iv: "a8159778fc82b68224974a5badc20aef"
server_key_version: "encryptionkey@0"
I20260809 08:42:22.556360 5604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:22.565218 5604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:22.567322 5604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:22.567502 5604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:22.571981 5604 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:33271
I20260809 08:42:22.572011 5697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:33271 every 8 connection(s)
I20260809 08:42:22.572316 5604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:22.573443 5698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:22.575901 5698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00: Bootstrap starting.
I20260809 08:42:22.576540 5698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:22.576871 5698 log.cc:824] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:22.577633 5698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00: No bootstrap required, opened a new log
I20260809 08:42:22.578840 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 5604
I20260809 08:42:22.578971 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:22.579176 31997 external_mini_cluster.cc:1428] Setting key e77f67508f641ee4b5acfdd0ecb0a036
I20260809 08:42:22.579768 5698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2c026d11ed42f4a33ca6e3adca3f00" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33271 } }
I20260809 08:42:22.579984 5698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:22.580040 5698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b2c026d11ed42f4a33ca6e3adca3f00, State: Initialized, Role: FOLLOWER
I20260809 08:42:22.580170 5698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2c026d11ed42f4a33ca6e3adca3f00" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33271 } }
I20260809 08:42:22.580268 5698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:22.580312 5698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:22.580361 5698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:22.581040 5698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2c026d11ed42f4a33ca6e3adca3f00" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33271 } }
I20260809 08:42:22.581171 5698 leader_election.cc:302] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b2c026d11ed42f4a33ca6e3adca3f00; no voters:
I20260809 08:42:22.581350 5698 leader_election.cc:288] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:22.581406 5708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:22.581600 5708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [term 1 LEADER]: Becoming Leader. State: Replica: 7b2c026d11ed42f4a33ca6e3adca3f00, State: Running, Role: LEADER
I20260809 08:42:22.581694 5698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:22.581745 5708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2c026d11ed42f4a33ca6e3adca3f00" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33271 } }
I20260809 08:42:22.582275 5711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b2c026d11ed42f4a33ca6e3adca3f00. Latest consensus state: current_term: 1 leader_uuid: "7b2c026d11ed42f4a33ca6e3adca3f00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2c026d11ed42f4a33ca6e3adca3f00" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33271 } } }
I20260809 08:42:22.582258 5710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b2c026d11ed42f4a33ca6e3adca3f00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b2c026d11ed42f4a33ca6e3adca3f00" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33271 } } }
I20260809 08:42:22.582422 5711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:22.582436 5710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:22.583113 5718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:22.583787 5718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:22.585799 5718 catalog_manager.cc:1347] Generated new cluster ID: a809f2eade6a4233b31fb61f28e67ffc
I20260809 08:42:22.585852 5718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:22.597121 5718 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:22.597772 5718 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 08:42:22 dist-test-slave-gkw7 krb5kdc[5585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:22.602798 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:22.579756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51834 (local address 127.31.63.126:33271)
0809 08:42:22.579950 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:22.579958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:22.579988 (+ 30us) server_negotiation.cc:407] Connection header received
0809 08:42:22.580438 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:22.580447 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:22.580510 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:22.580638 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:22.581544 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:22.583088 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:22.584126 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:22.584668 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:22.584795 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:22.598789 (+ 13994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:22.598793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:22.598794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:22.598807 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:22.600886 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:22.601298 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:22.601301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:22.601302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:22.601434 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:22.601675 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:22.601676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:22.601676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:22.601978 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:22.602128 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:22.602255 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 08:42:22.602403 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260809 08:42:22.608764 5718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00: Generated new TSK 0
I20260809 08:42:22.608952 5718 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:42:22.639091 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33271
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:22.731196 5731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:22.731436 5731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:22.731479 5731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:22.732679 5731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:22.732753 5731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:22.732800 5731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:22.734447 5731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:22.734722 5731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:22.734961 5731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:22.740310 5740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:22.740322 5745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:22.740377 5743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:22.740424 5731 server_base.cc:1034] running on GCE node
I20260809 08:42:22.740780 5731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:22.741000 5731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:22.742170 5731 hybrid_clock.cc:648] HybridClock initialized: now 1786264942742147 us; error 41 us; skew 500 ppm
Aug 09 08:42:22 dist-test-slave-gkw7 krb5kdc[5585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264942, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:22.744724 5731 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:42:22.745121 5731 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:22.745218 5731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:22.745383 5731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:22.746537 5731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "d2d121ab07ae4391b4497c50dceb2019"
format_stamp: "Formatted at 2026-08-09 08:42:22 on dist-test-slave-gkw7"
server_key: "6d83e2270ea248a1cf76c65841fd19f8"
server_key_iv: "eccc3db8dd964b153574e3acc959a7ab"
server_key_version: "encryptionkey@0"
I20260809 08:42:22.746917 5731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "d2d121ab07ae4391b4497c50dceb2019"
format_stamp: "Formatted at 2026-08-09 08:42:22 on dist-test-slave-gkw7"
server_key: "6d83e2270ea248a1cf76c65841fd19f8"
server_key_iv: "eccc3db8dd964b153574e3acc959a7ab"
server_key_version: "encryptionkey@0"
I20260809 08:42:22.748299 5731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:42:22.749049 5752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:22.749244 5731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:22.749346 5731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "d2d121ab07ae4391b4497c50dceb2019"
format_stamp: "Formatted at 2026-08-09 08:42:22 on dist-test-slave-gkw7"
server_key: "6d83e2270ea248a1cf76c65841fd19f8"
server_key_iv: "eccc3db8dd964b153574e3acc959a7ab"
server_key_version: "encryptionkey@0"
I20260809 08:42:22.749434 5731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:22.775574 5731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:22.777489 5731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:22.777684 5731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:22.777956 5731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:22.778303 5731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:22.778359 5731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:22.778406 5731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:22.778435 5731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:22.784993 5731 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:43897
I20260809 08:42:22.785039 5920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:43897 every 8 connection(s)
I20260809 08:42:22.785320 5731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:22.788893 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 5731
I20260809 08:42:22.789016 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:22.789196 31997 external_mini_cluster.cc:1428] Setting key 47a9c80d2488628be55cec726bd733d2
Aug 09 08:42:22 dist-test-slave-gkw7 krb5kdc[5585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264942, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:22.795646 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:22.786132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:42967 (local address 127.31.63.126:33271)
0809 08:42:22.786223 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:22.786225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:22.786444 (+ 219us) server_negotiation.cc:407] Connection header received
0809 08:42:22.786982 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:22.786984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:22.786998 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:22.787055 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:22.788098 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:22.788626 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:22.789974 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:22.790318 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:22.790398 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:22.792659 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:22.792660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:22.792661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:22.792669 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:22.794239 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:22.794652 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:22.794653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:22.794653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:22.794713 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:22.794974 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:22.794975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:22.794975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:22.795100 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:22.795181 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:22.795463 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 08:42:22.795551 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:42:22.795866 5924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:22.786115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33271 (local address 127.31.63.65:42967)
0809 08:42:22.786319 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:22.786356 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:22.786871 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:22.787124 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:22.787128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:22.787406 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:22.787991 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:22.787996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:22.788744 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:22.788745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:22.789880 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:22.789883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:22.790410 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:22.790411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:22.790514 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:22.791055 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:22.791064 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:22.792558 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:22.794323 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:22.794332 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:22.794334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:22.794563 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:22.794783 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:22.794783 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:22.794784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:22.794902 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:22.795171 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:22.795178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:22.795325 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 08:42:22.795439 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 08:42:22.795607 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260809 08:42:22.796422 5921 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33271
I20260809 08:42:22.796515 5921 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:22.796689 5921 heartbeater.cc:511] Master 127.31.63.126:33271 requested a full tablet report, sending...
I20260809 08:42:22.797305 5646 ts_manager.cc:194] Registered new tserver with Master: d2d121ab07ae4391b4497c50dceb2019 (127.31.63.65:43897)
I20260809 08:42:22.798298 5646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:42967
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:42:22.816968 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33271
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:22.900987 5928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:22.901211 5928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:22.901237 5928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:22.902460 5928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:22.902508 5928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:22.902534 5928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:22.904102 5928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:22.904374 5928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:22.904642 5928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:42:22.909897 5928 server_base.cc:1034] running on GCE node
W20260809 08:42:22.909891 5936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:22.909870 5935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:22.909878 5938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:22.910336 5928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:22.910540 5928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:22.911725 5928 hybrid_clock.cc:648] HybridClock initialized: now 1786264942911710 us; error 29 us; skew 500 ppm
Aug 09 08:42:22 dist-test-slave-gkw7 krb5kdc[5585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264942, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:22.914245 5928 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:42:22.914654 5928 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:22.914731 5928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:22.914884 5928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:22.915948 5928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "51b0f464857540de9c48489b4fed9221"
format_stamp: "Formatted at 2026-08-09 08:42:22 on dist-test-slave-gkw7"
server_key: "77be2391747a5ec3b36926492f5d5a88"
server_key_iv: "f68f564d274c6d01d4e7964f61448328"
server_key_version: "encryptionkey@0"
I20260809 08:42:22.916307 5928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "51b0f464857540de9c48489b4fed9221"
format_stamp: "Formatted at 2026-08-09 08:42:22 on dist-test-slave-gkw7"
server_key: "77be2391747a5ec3b36926492f5d5a88"
server_key_iv: "f68f564d274c6d01d4e7964f61448328"
server_key_version: "encryptionkey@0"
I20260809 08:42:22.917780 5928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:22.918594 5946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:22.918813 5928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:22.918921 5928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "51b0f464857540de9c48489b4fed9221"
format_stamp: "Formatted at 2026-08-09 08:42:22 on dist-test-slave-gkw7"
server_key: "77be2391747a5ec3b36926492f5d5a88"
server_key_iv: "f68f564d274c6d01d4e7964f61448328"
server_key_version: "encryptionkey@0"
I20260809 08:42:22.918999 5928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:22.934856 5928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:22.937107 5928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:22.937314 5928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:22.937661 5928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:22.938036 5928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:22.938098 5928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:22.938144 5928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:22.938175 5928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:22.945356 5928 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:37813
I20260809 08:42:22.945410 6116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:37813 every 8 connection(s)
I20260809 08:42:22.945791 5928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:42:22 dist-test-slave-gkw7 krb5kdc[5585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264942, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:22.952548 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 5928
I20260809 08:42:22.952641 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:42:22.952862 31997 external_mini_cluster.cc:1428] Setting key 5d9409bb5e5074e999430c63057770a2
I20260809 08:42:22.956384 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:22.946584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:38423 (local address 127.31.63.126:33271)
0809 08:42:22.946662 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:22.946665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:22.946845 (+ 180us) server_negotiation.cc:407] Connection header received
0809 08:42:22.947444 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:22.947445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:22.947460 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:22.947519 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:22.948590 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:22.949159 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:22.950377 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:22.950609 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:22.950705 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:22.953185 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:22.953187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:22.953188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:22.953196 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:22.954740 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:22.955271 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:22.955271 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:22.955272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:22.955327 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:22.955658 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:22.955659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:22.955660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:22.955831 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:22.955909 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:22.956164 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 08:42:22.956258 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 08:42:22.956480 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:22.946503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33271 (local address 127.31.63.66:38423)
0809 08:42:22.946691 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:22.946750 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:22.947318 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:22.947595 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:22.947600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:22.947909 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:22.948472 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:22.948478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:22.949267 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:22.949268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:22.950262 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:22.950265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:22.950719 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:22.950720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:22.950861 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:22.951365 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:22.951374 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:22.953056 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:22.954830 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:22.954839 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:22.954841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:22.955192 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:22.955422 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:22.955423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:22.955423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:22.955582 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:22.955902 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:22.955908 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:22.956031 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 08:42:22.956116 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:42:22.956277 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":76,"threads_started":1}
I20260809 08:42:22.956990 6118 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33271
I20260809 08:42:22.957089 6118 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:22.957285 6118 heartbeater.cc:511] Master 127.31.63.126:33271 requested a full tablet report, sending...
I20260809 08:42:22.957739 5646 ts_manager.cc:194] Registered new tserver with Master: 51b0f464857540de9c48489b4fed9221 (127.31.63.66:37813)
I20260809 08:42:22.958302 5646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:38423
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:42:22.980568 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33271
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:40663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:23.065815 6130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:23.066082 6130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:23.066125 6130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:23.067305 6130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:23.067375 6130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:23.067420 6130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:23.069096 6130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:23.069378 6130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:23.069654 6130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:23.075001 6137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:23.074965 6142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:23.074971 6140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:23.075234 6130 server_base.cc:1034] running on GCE node
I20260809 08:42:23.075497 6130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:23.075766 6130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:23.076937 6130 hybrid_clock.cc:648] HybridClock initialized: now 1786264943076912 us; error 38 us; skew 500 ppm
Aug 09 08:42:23 dist-test-slave-gkw7 krb5kdc[5585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264943, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:23.079697 6130 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:42:23.080025 6130 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:23.080071 6130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:23.080160 6130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:23.081182 6130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "af4e9c1ac7bc439c91e75cdeac888d3b"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "e5c5f86d8d407f98588fde4eb9d3f2ef"
server_key_iv: "5d8c35e530d007851887a72b01f157f7"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.081625 6130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "af4e9c1ac7bc439c91e75cdeac888d3b"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "e5c5f86d8d407f98588fde4eb9d3f2ef"
server_key_iv: "5d8c35e530d007851887a72b01f157f7"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.083144 6130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:23.084019 6148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.084198 6130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:23.084293 6130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "af4e9c1ac7bc439c91e75cdeac888d3b"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "e5c5f86d8d407f98588fde4eb9d3f2ef"
server_key_iv: "5d8c35e530d007851887a72b01f157f7"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.084349 6130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:23.098721 6130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:23.100627 6130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:23.100754 6130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:23.100982 6130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:23.101310 6130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:23.101346 6130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.101370 6130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:23.101382 6130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.107589 6130 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:41719
I20260809 08:42:23.107663 6308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:41719 every 8 connection(s)
I20260809 08:42:23.107962 6130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:42:23 dist-test-slave-gkw7 krb5kdc[5585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264943, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:23.116581 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 6130
I20260809 08:42:23.116691 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:23.116946 31997 external_mini_cluster.cc:1428] Setting key cfefd247a76a55b272a5f46493f9d8c5
I20260809 08:42:23.117504 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.108822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:51425 (local address 127.31.63.126:33271)
0809 08:42:23.108897 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.108899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.109118 (+ 219us) server_negotiation.cc:407] Connection header received
0809 08:42:23.109756 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.109756 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.109767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.109825 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:23.110791 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.111317 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.112291 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.112471 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.112521 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.114655 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:23.114656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:23.114656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:23.114663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:23.115910 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:23.116283 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:23.116284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:23.116284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:23.116336 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:23.116681 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:23.116682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:23.116683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:23.116823 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:23.116913 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.117285 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.117389 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:42:23.117760 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.108801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33271 (local address 127.31.63.67:51425)
0809 08:42:23.108967 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:23.109011 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:23.109651 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:23.109888 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:23.109891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:23.110168 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:23.110690 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.110696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.111399 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.111400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.112217 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.112219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.112546 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.112547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.112649 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.113130 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:23.113138 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:23.114568 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:23.115983 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:23.115991 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:23.115992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:23.116209 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:23.116438 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:23.116439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:23.116440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:23.116592 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:23.116901 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:23.116909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:23.117094 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 08:42:23.117248 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 08:42:23.117459 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260809 08:42:23.118194 6309 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33271
I20260809 08:42:23.118305 6309 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:23.118470 6309 heartbeater.cc:511] Master 127.31.63.126:33271 requested a full tablet report, sending...
I20260809 08:42:23.118837 5646 ts_manager.cc:194] Registered new tserver with Master: af4e9c1ac7bc439c91e75cdeac888d3b (127.31.63.67:41719)
I20260809 08:42:23.119210 5646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:51425
I20260809 08:42:23.128973 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:42:23.135969 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.130363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.31.63.126:33271)
0809 08:42:23.130417 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.130419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.130480 (+ 61us) server_negotiation.cc:407] Connection header received
0809 08:42:23.130523 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.130524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.130533 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.130586 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:23.131131 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.131613 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.132296 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.132473 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.132522 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.133651 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:23.133652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:23.133653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:23.133659 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:23.134969 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:23.135306 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:23.135306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:23.135307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:23.135358 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:23.135526 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:23.135527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:23.135527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:23.135669 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:23.135753 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.135845 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.135905 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:42:23.137734 5646 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:42:23.139350 5646 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:23.146996 6329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.142134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:37813 (local address 127.0.0.1:51958)
0809 08:42:23.142471 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:23.142481 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:23.142534 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:23.143062 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:23.143063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:23.143066 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:23.143101 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.143102 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.144383 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.144384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.145747 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.145749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.146672 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.146673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.146739 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.146750 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 08:42:23.146846 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 08:42:23.146909 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":4,"thread_start_us":108,"threads_started":1}
I20260809 08:42:23.147011 6328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.142279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51958 (local address 127.31.63.66:37813)
0809 08:42:23.142789 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.142793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.142805 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:42:23.142849 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.142851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.142868 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.142953 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:23.143192 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.144272 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.145841 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.146584 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.146664 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.146730 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.146870 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.146919 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":82,"threads_started":1}
I20260809 08:42:23.147913 6323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.141914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:41719 (local address 127.0.0.1:40830)
0809 08:42:23.142325 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:23.142373 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:23.142459 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:23.142871 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:23.142874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:23.142876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:23.142975 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.142977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.144176 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.144178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.145746 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.145749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.147648 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.147649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.147719 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.147723 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:42:23.147791 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 08:42:23.147839 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":54,"thread_start_us":82,"threads_started":1}
I20260809 08:42:23.147948 6324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.142045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.31.63.67:41719)
0809 08:42:23.142289 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.142295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.142571 (+ 276us) server_negotiation.cc:407] Connection header received
0809 08:42:23.142628 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.142631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.142649 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.142734 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:23.143085 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.144052 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.146771 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.147560 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.147628 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.147680 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.147817 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.147859 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":162,"threads_started":1}
I20260809 08:42:23.147833 6021 tablet_service.cc:1467] Processing CreateTablet for tablet 39fc990aba7945f9aa921446c2740c7b (DEFAULT_TABLE table=test-table [id=e525c0d1cc764da1b8a01aaf927642ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:23.148294 6021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39fc990aba7945f9aa921446c2740c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:23.148532 6326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.142346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:43897 (local address 127.0.0.1:45292)
0809 08:42:23.142589 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:23.142596 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:23.142651 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:23.142973 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:23.142974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:23.142976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:23.143025 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.143026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.145596 (+ 2570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.145597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.146648 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.146651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.148301 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.148301 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.148379 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.148383 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:42:23.148426 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:42:23.148467 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":65,"threads_started":1}
I20260809 08:42:23.148777 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.142485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.31.63.65:43897)
0809 08:42:23.142719 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.142724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.142733 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:42:23.142784 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.142786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.142802 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.142878 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:23.143106 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.144272 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.146755 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.148228 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.148302 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.148443 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.148469 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.148506 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":122,"threads_started":1}
I20260809 08:42:23.148677 6212 tablet_service.cc:1467] Processing CreateTablet for tablet 39fc990aba7945f9aa921446c2740c7b (DEFAULT_TABLE table=test-table [id=e525c0d1cc764da1b8a01aaf927642ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:23.149021 6212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39fc990aba7945f9aa921446c2740c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:23.149384 5815 tablet_service.cc:1467] Processing CreateTablet for tablet 39fc990aba7945f9aa921446c2740c7b (DEFAULT_TABLE table=test-table [id=e525c0d1cc764da1b8a01aaf927642ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:23.149709 5815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39fc990aba7945f9aa921446c2740c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:23.152405 6333 tablet_bootstrap.cc:492] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: Bootstrap starting.
I20260809 08:42:23.152670 6334 tablet_bootstrap.cc:492] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019: Bootstrap starting.
I20260809 08:42:23.153297 6333 tablet_bootstrap.cc:654] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:23.153298 6334 tablet_bootstrap.cc:654] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:23.153378 6335 tablet_bootstrap.cc:492] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: Bootstrap starting.
I20260809 08:42:23.153992 6334 log.cc:824] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:23.154126 6335 tablet_bootstrap.cc:654] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:23.154522 6333 log.cc:824] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:23.154810 6335 log.cc:824] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:23.155025 6334 tablet_bootstrap.cc:492] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019: No bootstrap required, opened a new log
I20260809 08:42:23.155115 6334 ts_tablet_manager.cc:1397] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:23.155471 6333 tablet_bootstrap.cc:492] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: No bootstrap required, opened a new log
I20260809 08:42:23.155543 6333 ts_tablet_manager.cc:1397] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:42:23.155627 6335 tablet_bootstrap.cc:492] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: No bootstrap required, opened a new log
I20260809 08:42:23.155710 6335 ts_tablet_manager.cc:1397] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:23.157222 6335 raft_consensus.cc:348] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 } } peers { permanent_uuid: "51b0f464857540de9c48489b4fed9221" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37813 } } peers { permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 } }
I20260809 08:42:23.157258 6334 raft_consensus.cc:348] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 } } peers { permanent_uuid: "51b0f464857540de9c48489b4fed9221" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37813 } } peers { permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 } }
I20260809 08:42:23.157402 6335 raft_consensus.cc:374] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:23.157416 6334 raft_consensus.cc:374] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:23.157454 6334 raft_consensus.cc:729] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2d121ab07ae4391b4497c50dceb2019, State: Initialized, Role: FOLLOWER
I20260809 08:42:23.157455 6335 raft_consensus.cc:729] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af4e9c1ac7bc439c91e75cdeac888d3b, State: Initialized, Role: FOLLOWER
I20260809 08:42:23.157634 6334 consensus_queue.cc:260] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 } } peers { permanent_uuid: "51b0f464857540de9c48489b4fed9221" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37813 } } peers { permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 } }
I20260809 08:42:23.157620 6335 consensus_queue.cc:260] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 } } peers { permanent_uuid: "51b0f464857540de9c48489b4fed9221" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37813 } } peers { permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 } }
I20260809 08:42:23.157615 6333 raft_consensus.cc:348] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 } } peers { permanent_uuid: "51b0f464857540de9c48489b4fed9221" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37813 } } peers { permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 } }
I20260809 08:42:23.157800 6333 raft_consensus.cc:374] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:23.157840 6333 raft_consensus.cc:729] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b0f464857540de9c48489b4fed9221, State: Initialized, Role: FOLLOWER
I20260809 08:42:23.157958 6335 ts_tablet_manager.cc:1428] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:42:23.157949 6333 consensus_queue.cc:260] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 } } peers { permanent_uuid: "51b0f464857540de9c48489b4fed9221" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37813 } } peers { permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 } }
I20260809 08:42:23.157991 6334 ts_tablet_manager.cc:1428] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:42:23.158144 6309 heartbeater.cc:503] Master 127.31.63.126:33271 was elected leader, sending a full tablet report...
I20260809 08:42:23.158145 5921 heartbeater.cc:503] Master 127.31.63.126:33271 was elected leader, sending a full tablet report...
I20260809 08:42:23.158207 6333 ts_tablet_manager.cc:1428] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:42:23.158438 6118 heartbeater.cc:503] Master 127.31.63.126:33271 was elected leader, sending a full tablet report...
W20260809 08:42:23.196415 6119 tablet.cc:2369] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:42:23.286100 5923 tablet.cc:2369] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:42:23.350028 6343 raft_consensus.cc:482] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:23.350229 6343 raft_consensus.cc:504] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 } } peers { permanent_uuid: "51b0f464857540de9c48489b4fed9221" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37813 } } peers { permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 } }
I20260809 08:42:23.350638 6343 leader_election.cc:288] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af4e9c1ac7bc439c91e75cdeac888d3b (127.31.63.67:41719), d2d121ab07ae4391b4497c50dceb2019 (127.31.63.65:43897)
I20260809 08:42:23.354125 6345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.350961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:43897 (local address 127.31.63.66:42105)
0809 08:42:23.351193 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:23.351204 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:23.351275 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:23.351441 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:23.351442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:23.351444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:23.351542 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.351544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.352408 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.352409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.353265 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.353267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.353829 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.353830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.353928 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.353932 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:42:23.353996 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 08:42:23.354037 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":132,"threads_started":1}
I20260809 08:42:23.354115 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.351143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:42105 (local address 127.31.63.65:43897)
0809 08:42:23.351204 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.351207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.351280 (+ 73us) server_negotiation.cc:407] Connection header received
0809 08:42:23.351346 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.351347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.351357 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.351436 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:23.351622 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.352316 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.353348 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.353740 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.353828 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.353838 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.354014 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.354051 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:42:23.354228 6324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.350954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:35337 (local address 127.31.63.67:41719)
0809 08:42:23.351020 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.351023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.351036 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:42:23.351123 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.351125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.351141 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.351196 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:23.351482 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.352260 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.353322 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.353807 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.353905 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.353914 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.354113 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.354151 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:42:23.354221 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.350858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:41719 (local address 127.31.63.66:35337)
0809 08:42:23.350921 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:23.350951 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:23.351024 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:23.351290 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:23.351291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:23.351293 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:23.351392 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.351394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.352345 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.352350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.353225 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:23.353228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.353939 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.353941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:23.354021 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.354024 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:42:23.354104 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:42:23.354146 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 08:42:23.354736 6242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39fc990aba7945f9aa921446c2740c7b" candidate_uuid: "51b0f464857540de9c48489b4fed9221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" is_pre_election: true
I20260809 08:42:23.354763 5844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39fc990aba7945f9aa921446c2740c7b" candidate_uuid: "51b0f464857540de9c48489b4fed9221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2d121ab07ae4391b4497c50dceb2019" is_pre_election: true
I20260809 08:42:23.354902 6242 raft_consensus.cc:2454] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0f464857540de9c48489b4fed9221 in term 0.
I20260809 08:42:23.354950 5844 raft_consensus.cc:2454] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51b0f464857540de9c48489b4fed9221 in term 0.
I20260809 08:42:23.355154 5955 leader_election.cc:302] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0f464857540de9c48489b4fed9221, af4e9c1ac7bc439c91e75cdeac888d3b; no voters:
I20260809 08:42:23.355310 6343 raft_consensus.cc:2781] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:23.355371 6343 raft_consensus.cc:482] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:23.355396 6343 raft_consensus.cc:3037] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:23.356154 6343 raft_consensus.cc:504] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 } } peers { permanent_uuid: "51b0f464857540de9c48489b4fed9221" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37813 } } peers { permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 } }
I20260809 08:42:23.356298 6343 leader_election.cc:288] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [CANDIDATE]: Term 1 election: Requested vote from peers af4e9c1ac7bc439c91e75cdeac888d3b (127.31.63.67:41719), d2d121ab07ae4391b4497c50dceb2019 (127.31.63.65:43897)
I20260809 08:42:23.356465 6242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39fc990aba7945f9aa921446c2740c7b" candidate_uuid: "51b0f464857540de9c48489b4fed9221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b"
I20260809 08:42:23.356514 5844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39fc990aba7945f9aa921446c2740c7b" candidate_uuid: "51b0f464857540de9c48489b4fed9221" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2d121ab07ae4391b4497c50dceb2019"
I20260809 08:42:23.356580 6242 raft_consensus.cc:3037] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:23.356609 5844 raft_consensus.cc:3037] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:23.357317 5844 raft_consensus.cc:2454] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0f464857540de9c48489b4fed9221 in term 1.
I20260809 08:42:23.357369 6242 raft_consensus.cc:2454] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51b0f464857540de9c48489b4fed9221 in term 1.
I20260809 08:42:23.357568 5955 leader_election.cc:302] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51b0f464857540de9c48489b4fed9221, af4e9c1ac7bc439c91e75cdeac888d3b; no voters:
I20260809 08:42:23.357676 6343 raft_consensus.cc:2781] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:23.357859 6343 raft_consensus.cc:686] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 1 LEADER]: Becoming Leader. State: Replica: 51b0f464857540de9c48489b4fed9221, State: Running, Role: LEADER
I20260809 08:42:23.357960 6343 consensus_queue.cc:237] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 } } peers { permanent_uuid: "51b0f464857540de9c48489b4fed9221" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37813 } } peers { permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 } }
W20260809 08:42:23.358608 6311 tablet.cc:2369] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:42:23.358877 5644 catalog_manager.cc:5495] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51b0f464857540de9c48489b4fed9221 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "51b0f464857540de9c48489b4fed9221" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51b0f464857540de9c48489b4fed9221" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 } health_report { overall_health: UNKNOWN } } }
I20260809 08:42:23.373133 6328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.370888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51964 (local address 127.31.63.66:37813)
0809 08:42:23.370954 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.370956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.370964 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:23.371025 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.371026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.371046 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.371099 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:23.371233 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.371707 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.372425 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:23.372605 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:23.372655 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:23.372799 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:23.372899 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:23.372960 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.372987 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.373005 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:42:23.375174 5844 raft_consensus.cc:1261] T 39fc990aba7945f9aa921446c2740c7b P d2d121ab07ae4391b4497c50dceb2019 [term 1 FOLLOWER]: Refusing update from remote peer 51b0f464857540de9c48489b4fed9221: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:23.375175 6242 raft_consensus.cc:1261] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b [term 1 FOLLOWER]: Refusing update from remote peer 51b0f464857540de9c48489b4fed9221: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:23.375566 6346 consensus_queue.cc:1035] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af4e9c1ac7bc439c91e75cdeac888d3b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:23.375741 6343 consensus_queue.cc:1035] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2d121ab07ae4391b4497c50dceb2019" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 43897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:23.378633 6350 mvcc.cc:205] Tried to move back new op lower bound from 7316541208061616128 to 7316541207994839040. Current Snapshot: MvccSnapshot[applied={T|T < 7316541208061616128}]
I20260809 08:42:23.378931 6349 mvcc.cc:205] Tried to move back new op lower bound from 7316541208061616128 to 7316541207994839040. Current Snapshot: MvccSnapshot[applied={T|T < 7316541208061616128 or (T in {7316541208061616128})}]
I20260809 08:42:23.379330 6353 mvcc.cc:205] Tried to move back new op lower bound from 7316541208061616128 to 7316541207994839040. Current Snapshot: MvccSnapshot[applied={T|T < 7316541208061616128}]
I20260809 08:42:23.380901 5645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:23.380996 5645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:23.381840 5645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b2c026d11ed42f4a33ca6e3adca3f00: Sending DeleteTablet for 3 replicas of tablet 39fc990aba7945f9aa921446c2740c7b
I20260809 08:42:23.382267 6212 tablet_service.cc:1514] Processing DeleteTablet for tablet 39fc990aba7945f9aa921446c2740c7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:23 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:40830
I20260809 08:42:23.382323 5815 tablet_service.cc:1514] Processing DeleteTablet for tablet 39fc990aba7945f9aa921446c2740c7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:23 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:45292
I20260809 08:42:23.382319 6021 tablet_service.cc:1514] Processing DeleteTablet for tablet 39fc990aba7945f9aa921446c2740c7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:23 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:51958
I20260809 08:42:23.382647 6364 tablet_replica.cc:330] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: stopping tablet replica
I20260809 08:42:23.382762 6364 raft_consensus.cc:2229] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:23.382822 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 5731
I20260809 08:42:23.410877 6366 tablet_replica.cc:330] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: stopping tablet replica
I20260809 08:42:23.411067 6366 raft_consensus.cc:2229] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:42:23.411216 6366 raft_consensus.cc:2258] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:23.382872 6364 raft_consensus.cc:2258] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:23.411811 6366 ts_tablet_manager.cc:1905] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:23.413401 6366 ts_tablet_manager.cc:1918] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:42:23.413502 6366 log.cc:1196] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/39fc990aba7945f9aa921446c2740c7b
I20260809 08:42:23.413932 6366 ts_tablet_manager.cc:1939] T 39fc990aba7945f9aa921446c2740c7b P 51b0f464857540de9c48489b4fed9221: Deleting consensus metadata
I20260809 08:42:23.414419 5630 catalog_manager.cc:4841] TS 51b0f464857540de9c48489b4fed9221 (127.31.63.66:37813): tablet 39fc990aba7945f9aa921446c2740c7b (table test-table [id=e525c0d1cc764da1b8a01aaf927642ca]) successfully deleted
W20260809 08:42:23.416985 5630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:43897 (error 108)
I20260809 08:42:23.417407 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 5928
I20260809 08:42:23.417519 6326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.417384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:43897 (local address 127.0.0.1:45296)
0809 08:42:23.417440 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:23.417481 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:43897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 08:42:23.432121 5630 catalog_manager.cc:4584] TS d2d121ab07ae4391b4497c50dceb2019 (127.31.63.65:43897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 39fc990aba7945f9aa921446c2740c7b: Network error: Client connection negotiation failed: client connection to 127.31.63.65:43897: connect: Connection refused (error 111)
I20260809 08:42:23.436029 6364 ts_tablet_manager.cc:1905] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:23.437733 6364 ts_tablet_manager.cc:1918] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:42:23.437819 6364 log.cc:1196] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/39fc990aba7945f9aa921446c2740c7b
I20260809 08:42:23.438105 6364 ts_tablet_manager.cc:1939] T 39fc990aba7945f9aa921446c2740c7b P af4e9c1ac7bc439c91e75cdeac888d3b: Deleting consensus metadata
I20260809 08:42:23.438630 5630 catalog_manager.cc:4841] TS af4e9c1ac7bc439c91e75cdeac888d3b (127.31.63.67:41719): tablet 39fc990aba7945f9aa921446c2740c7b (table test-table [id=e525c0d1cc764da1b8a01aaf927642ca]) successfully deleted
I20260809 08:42:23.440980 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 6130
I20260809 08:42:23.456082 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 5604
2026-08-09T08:42:23Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2383 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T08:42:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:23Z Disabled control of system clock
I20260809 08:42:23.492789 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:42937
--webserver_interface=127.31.63.126
--webserver_port=0
--builtin_ntp_servers=127.31.63.84:46313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:42937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 08:42:23.577723 6376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:23.577930 6376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:23.577951 6376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:23.579030 6376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:23.579063 6376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:23.579077 6376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:23.579089 6376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:23.579100 6376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:23.580382 6376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:42937
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:42937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:23.580585 6376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:23.580835 6376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:23.585856 6386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:23.585856 6391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:23.586001 6376 server_base.cc:1034] running on GCE node
W20260809 08:42:23.586102 6387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:23.586362 6376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:23.586655 6376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:23.587809 6376 hybrid_clock.cc:648] HybridClock initialized: now 1786264943587791 us; error 41 us; skew 500 ppm
I20260809 08:42:23.589077 6376 webserver.cc:460] Webserver started at http://127.31.63.126:36451/ using document root <none> and password file <none>
I20260809 08:42:23.589366 6376 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:23.589414 6376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:23.589617 6376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:23.590701 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "eb5ae6de700e46159222fe4398aeb987"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "7d459b6613b32d0bf260db716b84a021"
server_key_iv: "a8fabc19ae66e0413ef2c6e5e50b4c48"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.591084 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "eb5ae6de700e46159222fe4398aeb987"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "7d459b6613b32d0bf260db716b84a021"
server_key_iv: "a8fabc19ae66e0413ef2c6e5e50b4c48"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.592379 6376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:23.593086 6401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.593273 6376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:23.593364 6376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "eb5ae6de700e46159222fe4398aeb987"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "7d459b6613b32d0bf260db716b84a021"
server_key_iv: "a8fabc19ae66e0413ef2c6e5e50b4c48"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.593437 6376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:23.596817 6376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:23.597085 6376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:23.597230 6376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:23.601202 6376 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:42937
I20260809 08:42:23.601256 6478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:42937 every 8 connection(s)
I20260809 08:42:23.601611 6376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:23.602355 6481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:23.605204 6481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987: Bootstrap starting.
I20260809 08:42:23.605870 6481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:23.606169 6481 log.cc:824] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:23.606863 6481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987: No bootstrap required, opened a new log
I20260809 08:42:23.607635 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 6376
I20260809 08:42:23.607738 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:23.607961 31997 external_mini_cluster.cc:1428] Setting key 576fb14c39990721d84af15b41ae8a0b
I20260809 08:42:23.609090 6481 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae6de700e46159222fe4398aeb987" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42937 } }
I20260809 08:42:23.609302 6481 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:23.609344 6481 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5ae6de700e46159222fe4398aeb987, State: Initialized, Role: FOLLOWER
I20260809 08:42:23.609474 6481 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae6de700e46159222fe4398aeb987" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42937 } }
I20260809 08:42:23.609581 6481 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:23.609625 6481 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:23.609684 6481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:23.610435 6481 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae6de700e46159222fe4398aeb987" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42937 } }
I20260809 08:42:23.610491 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.608503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.31.63.126:42937)
0809 08:42:23.608769 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.608780 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.608818 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:42:23.609046 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.609054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.609116 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.609276 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:23.610046 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:23.610049 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:23.610050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:23.610054 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:23.610093 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:23.610106 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:23.610182 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.610220 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.610277 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
I20260809 08:42:23.610544 6481 leader_election.cc:302] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb5ae6de700e46159222fe4398aeb987; no voters:
I20260809 08:42:23.610714 6481 leader_election.cc:288] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:23.610985 6481 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:23.611330 6489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:23.611474 6489 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [term 1 LEADER]: Becoming Leader. State: Replica: eb5ae6de700e46159222fe4398aeb987, State: Running, Role: LEADER
I20260809 08:42:23.611579 6489 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae6de700e46159222fe4398aeb987" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42937 } }
I20260809 08:42:23.612035 6493 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb5ae6de700e46159222fe4398aeb987. Latest consensus state: current_term: 1 leader_uuid: "eb5ae6de700e46159222fe4398aeb987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae6de700e46159222fe4398aeb987" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42937 } } }
I20260809 08:42:23.612147 6493 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:23.612182 6490 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb5ae6de700e46159222fe4398aeb987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb5ae6de700e46159222fe4398aeb987" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42937 } } }
I20260809 08:42:23.612267 6490 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:23.612375 6506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:23.612887 6506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:23.614502 6506 catalog_manager.cc:1347] Generated new cluster ID: fe4e2540b11744de8d59543c412dfc9a
I20260809 08:42:23.614562 6506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:23.622464 6506 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:23.622632 6506 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:23.632817 6506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987: Generated new TSK 0
I20260809 08:42:23.633052 6506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:42:23.635547 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42937
--builtin_ntp_servers=127.31.63.84:46313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 08:42:23.739660 6517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:23.739917 6517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:23.739956 6517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:23.741159 6517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:23.741222 6517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:23.741267 6517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:23.742736 6517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:23.743017 6517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:23.743332 6517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:23.749356 6532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:23.749425 6531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:23.749581 6517 server_base.cc:1034] running on GCE node
W20260809 08:42:23.749666 6535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:23.749938 6517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:23.750209 6517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:23.751379 6517 hybrid_clock.cc:648] HybridClock initialized: now 1786264943751365 us; error 31 us; skew 500 ppm
I20260809 08:42:23.752660 6517 webserver.cc:460] Webserver started at http://127.31.63.65:41377/ using document root <none> and password file <none>
I20260809 08:42:23.752918 6517 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:23.752990 6517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:23.753154 6517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:23.754336 6517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "6f714369b46c482d87533ee372f6b7a4"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "f2020abbaf7771a3b88ad916aaf12844"
server_key_iv: "b3c92930083eb39dac90551ef3a57c1e"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.754706 6517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "6f714369b46c482d87533ee372f6b7a4"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "f2020abbaf7771a3b88ad916aaf12844"
server_key_iv: "b3c92930083eb39dac90551ef3a57c1e"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.756112 6517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:42:23.756947 6542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.757128 6517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:23.757253 6517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "6f714369b46c482d87533ee372f6b7a4"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "f2020abbaf7771a3b88ad916aaf12844"
server_key_iv: "b3c92930083eb39dac90551ef3a57c1e"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.757332 6517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:23.760743 6517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:23.761034 6517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:23.761193 6517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:23.761431 6517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:23.761838 6517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:23.761883 6517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.761925 6517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:23.761951 6517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.768146 6517 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:40939
I20260809 08:42:23.768239 6707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:40939 every 8 connection(s)
I20260809 08:42:23.768595 6517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:23.772030 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.769453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:47571 (local address 127.31.63.126:42937)
0809 08:42:23.769506 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.769508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.769828 (+ 320us) server_negotiation.cc:407] Connection header received
0809 08:42:23.770452 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.770453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.770460 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.770506 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:23.771747 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:23.771748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:23.771748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:23.771752 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:23.771768 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:23.771772 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:23.771815 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.771923 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.771942 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:42:23.772092 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 6517
I20260809 08:42:23.772131 6711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.769449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42937 (local address 127.31.63.65:47571)
0809 08:42:23.769674 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:23.769727 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:23.770347 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:23.770575 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:23.770579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:23.770589 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:23.771603 (+ 1014us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:42:23.771603 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:23.771632 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:42:23.771660 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:23.771820 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:23.771822 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:23.771825 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:42:23.771875 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:42:23.771924 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":118,"threads_started":1}
I20260809 08:42:23.772185 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:23.772454 31997 external_mini_cluster.cc:1428] Setting key d8282091855d5b8992a0f33c80db026e
I20260809 08:42:23.772691 6709 heartbeater.cc:347] Connected to a master server at 127.31.63.126:42937
I20260809 08:42:23.772810 6709 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:23.772962 6709 heartbeater.cc:511] Master 127.31.63.126:42937 requested a full tablet report, sending...
I20260809 08:42:23.773414 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42937
--builtin_ntp_servers=127.31.63.84:46313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 08:42:23.773602 6430 ts_manager.cc:194] Registered new tserver with Master: 6f714369b46c482d87533ee372f6b7a4 (127.31.63.65:40939)
W20260809 08:42:23.858132 6712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:23.858374 6712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:23.858413 6712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:23.859601 6712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:23.859669 6712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:23.859743 6712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:23.861338 6712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:23.861634 6712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:23.861977 6712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:23.867321 6719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:23.867277 6721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:23.867277 6723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:23.867784 6712 server_base.cc:1034] running on GCE node
I20260809 08:42:23.868032 6712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:23.868278 6712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:23.869450 6712 hybrid_clock.cc:648] HybridClock initialized: now 1786264943869434 us; error 40 us; skew 500 ppm
I20260809 08:42:23.870867 6712 webserver.cc:460] Webserver started at http://127.31.63.66:42633/ using document root <none> and password file <none>
I20260809 08:42:23.871122 6712 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:23.871181 6712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:23.871306 6712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:23.872524 6712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "90282f2dece84a7b8eaceb396ef16e23"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "ae85852b48e4060cbe6ea194c8e34927"
server_key_iv: "969d19ba4c4aa145db3a2e64b994a74d"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.872848 6712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "90282f2dece84a7b8eaceb396ef16e23"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "ae85852b48e4060cbe6ea194c8e34927"
server_key_iv: "969d19ba4c4aa145db3a2e64b994a74d"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.874233 6712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:42:23.874994 6732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.875200 6712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:23.875277 6712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "90282f2dece84a7b8eaceb396ef16e23"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "ae85852b48e4060cbe6ea194c8e34927"
server_key_iv: "969d19ba4c4aa145db3a2e64b994a74d"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.875332 6712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:23.878729 6712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:23.878957 6712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:23.879050 6712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:23.879211 6712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:23.879464 6712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:23.879493 6712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.879513 6712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:23.879526 6712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.885864 6712 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:46821
I20260809 08:42:23.885941 6900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:46821 every 8 connection(s)
I20260809 08:42:23.886291 6712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:42:23.888793 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 6712
I20260809 08:42:23.888916 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:42:23.889144 31997 external_mini_cluster.cc:1428] Setting key 84afaf0162ce2c2694448bbee2c9630d
I20260809 08:42:23.889853 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.887045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:55009 (local address 127.31.63.126:42937)
0809 08:42:23.887098 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:23.887099 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:23.887408 (+ 309us) server_negotiation.cc:407] Connection header received
0809 08:42:23.887952 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:23.887954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:23.887962 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:23.888015 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:23.889529 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:23.889531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:23.889531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:23.889536 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:23.889555 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:23.889579 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:23.889637 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:23.889759 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 08:42:23.889777 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:42:23.890002 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:23.887020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42937 (local address 127.31.63.66:55009)
0809 08:42:23.887271 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:23.887314 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:23.887832 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:23.888088 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:23.888092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:23.888102 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:23.889307 (+ 1205us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:42:23.889307 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:23.889348 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:42:23.889391 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:23.889667 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:23.889673 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:23.889675 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:42:23.889737 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:42:23.889789 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260809 08:42:23.890478 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42937
--builtin_ntp_servers=127.31.63.84:46313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 08:42:23.890609 6901 heartbeater.cc:347] Connected to a master server at 127.31.63.126:42937
I20260809 08:42:23.890736 6901 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:23.890926 6901 heartbeater.cc:511] Master 127.31.63.126:42937 requested a full tablet report, sending...
I20260809 08:42:23.891397 6430 ts_manager.cc:194] Registered new tserver with Master: 90282f2dece84a7b8eaceb396ef16e23 (127.31.63.66:46821)
W20260809 08:42:23.976881 6907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:23.977094 6907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:23.977118 6907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:23.978353 6907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:23.978394 6907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:23.978420 6907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:23.979775 6907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:23.979984 6907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:23.980247 6907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:42:23.985867 6907 server_base.cc:1034] running on GCE node
W20260809 08:42:23.985811 6917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:23.985821 6915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:23.985811 6914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:23.986254 6907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:23.986516 6907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:23.987684 6907 hybrid_clock.cc:648] HybridClock initialized: now 1786264943987672 us; error 40 us; skew 500 ppm
I20260809 08:42:23.989147 6907 webserver.cc:460] Webserver started at http://127.31.63.67:39935/ using document root <none> and password file <none>
I20260809 08:42:23.989419 6907 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:23.989494 6907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:23.989686 6907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:23.990805 6907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "63792240b63e4f4d86190d7d3d2dd3dc"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "1a58b23a3c9db665f6f009b0c7b3a709"
server_key_iv: "cebcae0f0e098baff08e8388226937fd"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.991189 6907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "63792240b63e4f4d86190d7d3d2dd3dc"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "1a58b23a3c9db665f6f009b0c7b3a709"
server_key_iv: "cebcae0f0e098baff08e8388226937fd"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.992566 6907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:42:23.993322 6924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.993502 6907 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:23.993654 6907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "63792240b63e4f4d86190d7d3d2dd3dc"
format_stamp: "Formatted at 2026-08-09 08:42:23 on dist-test-slave-gkw7"
server_key: "1a58b23a3c9db665f6f009b0c7b3a709"
server_key_iv: "cebcae0f0e098baff08e8388226937fd"
server_key_version: "encryptionkey@0"
I20260809 08:42:23.993738 6907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:23.997411 6907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:23.997750 6907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:23.997903 6907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:23.998135 6907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:23.998474 6907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:23.998530 6907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:23.998570 6907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:23.998598 6907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:24.004730 6907 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:37993
I20260809 08:42:24.004815 7074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:37993 every 8 connection(s)
I20260809 08:42:24.005144 6907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:42:24.006656 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 6907
I20260809 08:42:24.006786 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:24.006994 31997 external_mini_cluster.cc:1428] Setting key 3072981016b79c4fdcda239aed998d23
I20260809 08:42:24.008422 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.005949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:36911 (local address 127.31.63.126:42937)
0809 08:42:24.005997 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.005999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.006235 (+ 236us) server_negotiation.cc:407] Connection header received
0809 08:42:24.006759 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.006762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.006771 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.006852 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.008172 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.008173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.008174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.008179 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.008196 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.008201 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.008254 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.008331 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.008349 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:42:24.008627 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.005924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42937 (local address 127.31.63.67:36911)
0809 08:42:24.006118 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.006151 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:24.006639 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:24.006905 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:24.006908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:24.006916 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:24.008009 (+ 1093us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:42:24.008010 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:24.008039 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:42:24.008070 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:24.008253 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:24.008259 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:24.008262 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:42:24.008332 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:42:24.008398 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260809 08:42:24.009088 7076 heartbeater.cc:347] Connected to a master server at 127.31.63.126:42937
I20260809 08:42:24.009171 7076 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:24.009392 7076 heartbeater.cc:511] Master 127.31.63.126:42937 requested a full tablet report, sending...
I20260809 08:42:24.009788 6430 ts_manager.cc:194] Registered new tserver with Master: 63792240b63e4f4d86190d7d3d2dd3dc (127.31.63.67:37993)
I20260809 08:42:24.018635 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:42:24.021143 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.020145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.31.63.126:42937)
0809 08:42:24.020232 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.020234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.020370 (+ 136us) server_negotiation.cc:407] Connection header received
0809 08:42:24.020398 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.020399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.020404 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.020458 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.020969 (+ 511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.020970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.020970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.020974 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.020990 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.020995 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.021044 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.021079 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.021091 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 08:42:24.022878 6430 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:42:24.024398 6430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:24.028455 7095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.026606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:46821 (local address 127.0.0.1:55650)
0809 08:42:24.026781 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.026824 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:24.026884 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:24.027539 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:24.027540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:24.027543 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:24.027983 (+ 440us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:42:24.027984 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:24.028001 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:42:24.028026 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:24.028280 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:24.028289 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:24.028291 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:42:24.028362 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:42:24.028376 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":111,"threads_started":1}
I20260809 08:42:24.028508 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.026697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55650 (local address 127.31.63.66:46821)
0809 08:42:24.027074 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.027078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.027089 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:42:24.027118 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.027121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.027132 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.027220 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.028120 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.028129 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.028130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.028135 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.028172 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.028199 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.028270 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.028379 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.028393 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":102,"threads_started":1}
I20260809 08:42:24.028720 7099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.026607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:37993 (local address 127.0.0.1:53732)
0809 08:42:24.027062 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.027072 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:24.027151 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:24.027451 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:24.027459 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:24.027466 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:24.028046 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:42:24.028046 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:24.028054 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:42:24.028068 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:24.028610 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:24.028611 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:24.028612 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:42:24.028645 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 08:42:24.028660 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":3,"thread_start_us":71,"threads_started":1}
I20260809 08:42:24.028947 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.026835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:40939 (local address 127.0.0.1:38064)
0809 08:42:24.027250 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.027259 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:24.027294 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:24.028143 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:24.028144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:24.028146 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:24.028523 (+ 377us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:42:24.028524 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:24.028531 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:42:24.028542 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:24.028803 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:24.028804 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:24.028804 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 08:42:24.028844 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 08:42:24.028856 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":73,"threads_started":1}
I20260809 08:42:24.029012 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.026797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.31.63.67:37993)
0809 08:42:24.027003 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.027008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.027316 (+ 308us) server_negotiation.cc:407] Connection header received
0809 08:42:24.027347 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.027350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.027360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.027429 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.028164 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.028172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.028172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.028177 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.028224 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.028519 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.028889 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.028914 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.028930 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":114,"threads_started":1}
I20260809 08:42:24.029209 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.027080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38064 (local address 127.31.63.65:40939)
0809 08:42:24.027737 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.027740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.027748 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:24.027770 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.027771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.027778 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.027867 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.028630 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.028640 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.028640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.028645 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.028707 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.028732 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.028781 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.029104 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.029121 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":96,"threads_started":1}
I20260809 08:42:24.029305 6800 tablet_service.cc:1467] Processing CreateTablet for tablet 9b708a7eb2894e37864d1fd907904e82 (DEFAULT_TABLE table=test-table [id=95505854e2a64fe696dd43a61afee5a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:24.029762 6800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b708a7eb2894e37864d1fd907904e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:24.029700 6992 tablet_service.cc:1467] Processing CreateTablet for tablet 9b708a7eb2894e37864d1fd907904e82 (DEFAULT_TABLE table=test-table [id=95505854e2a64fe696dd43a61afee5a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:24.029825 6614 tablet_service.cc:1467] Processing CreateTablet for tablet 9b708a7eb2894e37864d1fd907904e82 (DEFAULT_TABLE table=test-table [id=95505854e2a64fe696dd43a61afee5a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:24.030021 6992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b708a7eb2894e37864d1fd907904e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:24.030203 6614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b708a7eb2894e37864d1fd907904e82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:24.032687 7102 tablet_bootstrap.cc:492] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4: Bootstrap starting.
I20260809 08:42:24.033248 7106 tablet_bootstrap.cc:492] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: Bootstrap starting.
I20260809 08:42:24.033663 7102 tablet_bootstrap.cc:654] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:24.034004 7106 tablet_bootstrap.cc:654] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:24.034020 7102 log.cc:824] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:24.034363 7106 log.cc:824] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:24.034775 7103 tablet_bootstrap.cc:492] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23: Bootstrap starting.
I20260809 08:42:24.035007 7102 tablet_bootstrap.cc:492] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4: No bootstrap required, opened a new log
I20260809 08:42:24.035080 7106 tablet_bootstrap.cc:492] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: No bootstrap required, opened a new log
I20260809 08:42:24.035130 7102 ts_tablet_manager.cc:1397] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 08:42:24.035168 7106 ts_tablet_manager.cc:1397] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:42:24.035424 7103 tablet_bootstrap.cc:654] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:24.035776 7103 log.cc:824] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:24.037302 7102 raft_consensus.cc:348] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 } } peers { permanent_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37993 } } peers { permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 } }
I20260809 08:42:24.037302 7106 raft_consensus.cc:348] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 } } peers { permanent_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37993 } } peers { permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 } }
I20260809 08:42:24.037436 7102 raft_consensus.cc:374] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:24.037460 7102 raft_consensus.cc:729] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f714369b46c482d87533ee372f6b7a4, State: Initialized, Role: FOLLOWER
I20260809 08:42:24.037465 7106 raft_consensus.cc:374] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:24.037508 7106 raft_consensus.cc:729] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63792240b63e4f4d86190d7d3d2dd3dc, State: Initialized, Role: FOLLOWER
I20260809 08:42:24.037667 7102 consensus_queue.cc:260] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 } } peers { permanent_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37993 } } peers { permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 } }
I20260809 08:42:24.037663 7106 consensus_queue.cc:260] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 } } peers { permanent_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37993 } } peers { permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 } }
I20260809 08:42:24.037966 7106 ts_tablet_manager.cc:1428] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 08:42:24.037976 7102 ts_tablet_manager.cc:1428] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:42:24.038113 7076 heartbeater.cc:503] Master 127.31.63.126:42937 was elected leader, sending a full tablet report...
I20260809 08:42:24.038244 7103 tablet_bootstrap.cc:492] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23: No bootstrap required, opened a new log
I20260809 08:42:24.038314 7103 ts_tablet_manager.cc:1397] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 08:42:24.038537 6709 heartbeater.cc:503] Master 127.31.63.126:42937 was elected leader, sending a full tablet report...
I20260809 08:42:24.040529 7103 raft_consensus.cc:348] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 } } peers { permanent_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37993 } } peers { permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 } }
I20260809 08:42:24.040676 7103 raft_consensus.cc:374] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:24.040736 7103 raft_consensus.cc:729] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90282f2dece84a7b8eaceb396ef16e23, State: Initialized, Role: FOLLOWER
I20260809 08:42:24.040850 7103 consensus_queue.cc:260] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 } } peers { permanent_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37993 } } peers { permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 } }
I20260809 08:42:24.041100 7103 ts_tablet_manager.cc:1428] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:42:24.041157 6901 heartbeater.cc:503] Master 127.31.63.126:42937 was elected leader, sending a full tablet report...
W20260809 08:42:24.136886 6904 tablet.cc:2369] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:42:24.197280 7115 raft_consensus.cc:482] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:24.197466 7115 raft_consensus.cc:504] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 } } peers { permanent_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37993 } } peers { permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 } }
I20260809 08:42:24.197950 7115 leader_election.cc:288] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90282f2dece84a7b8eaceb396ef16e23 (127.31.63.66:46821), 6f714369b46c482d87533ee372f6b7a4 (127.31.63.65:40939)
I20260809 08:42:24.199298 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.198178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:40939 (local address 127.31.63.67:53757)
0809 08:42:24.198242 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.198256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:24.198309 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:24.198477 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:24.198478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:24.198481 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:24.198918 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:42:24.198919 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:24.198926 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:42:24.198938 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:24.199139 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:24.199140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:24.199140 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 08:42:24.199195 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:42:24.199209 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 08:42:24.199342 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.198268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:53757 (local address 127.31.63.65:40939)
0809 08:42:24.198317 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.198320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.198359 (+ 39us) server_negotiation.cc:407] Connection header received
0809 08:42:24.198389 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.198392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.198404 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.198446 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.199034 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.199036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.199036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.199041 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.199059 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.199064 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.199142 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.199233 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.199268 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:42:24.199596 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.198296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:34839 (local address 127.31.63.66:46821)
0809 08:42:24.198342 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.198344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.198496 (+ 152us) server_negotiation.cc:407] Connection header received
0809 08:42:24.198527 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.198528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.198535 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.198583 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.199306 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.199307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.199307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.199312 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.199328 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.199332 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.199393 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.199517 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.199534 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:42:24.199599 7118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.198188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:46821 (local address 127.31.63.67:34839)
0809 08:42:24.198401 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.198408 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:24.198474 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:24.198587 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:24.198587 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:24.198589 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:24.199193 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:42:24.199194 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:24.199200 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:42:24.199212 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:24.199425 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:24.199426 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:24.199427 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:42:24.199505 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:42:24.199519 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":21,"thread_start_us":101,"threads_started":1}
I20260809 08:42:24.199676 6643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b708a7eb2894e37864d1fd907904e82" candidate_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f714369b46c482d87533ee372f6b7a4" is_pre_election: true
I20260809 08:42:24.199843 6643 raft_consensus.cc:2454] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63792240b63e4f4d86190d7d3d2dd3dc in term 0.
I20260809 08:42:24.199942 6835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b708a7eb2894e37864d1fd907904e82" candidate_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90282f2dece84a7b8eaceb396ef16e23" is_pre_election: true
I20260809 08:42:24.200088 6835 raft_consensus.cc:2454] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63792240b63e4f4d86190d7d3d2dd3dc in term 0.
I20260809 08:42:24.200140 6928 leader_election.cc:302] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63792240b63e4f4d86190d7d3d2dd3dc, 6f714369b46c482d87533ee372f6b7a4; no voters:
I20260809 08:42:24.200310 7115 raft_consensus.cc:2781] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:24.200379 7115 raft_consensus.cc:482] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:24.200416 7115 raft_consensus.cc:3037] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:24.201171 7115 raft_consensus.cc:504] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 } } peers { permanent_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37993 } } peers { permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 } }
I20260809 08:42:24.201327 7115 leader_election.cc:288] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [CANDIDATE]: Term 1 election: Requested vote from peers 90282f2dece84a7b8eaceb396ef16e23 (127.31.63.66:46821), 6f714369b46c482d87533ee372f6b7a4 (127.31.63.65:40939)
I20260809 08:42:24.201498 6643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b708a7eb2894e37864d1fd907904e82" candidate_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f714369b46c482d87533ee372f6b7a4"
I20260809 08:42:24.201498 6835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b708a7eb2894e37864d1fd907904e82" candidate_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90282f2dece84a7b8eaceb396ef16e23"
I20260809 08:42:24.201607 6643 raft_consensus.cc:3037] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:24.201608 6835 raft_consensus.cc:3037] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:24.202558 6835 raft_consensus.cc:2454] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63792240b63e4f4d86190d7d3d2dd3dc in term 1.
I20260809 08:42:24.202558 6643 raft_consensus.cc:2454] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63792240b63e4f4d86190d7d3d2dd3dc in term 1.
I20260809 08:42:24.202744 6927 leader_election.cc:302] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63792240b63e4f4d86190d7d3d2dd3dc, 90282f2dece84a7b8eaceb396ef16e23; no voters:
I20260809 08:42:24.202862 7115 raft_consensus.cc:2781] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:24.203045 7115 raft_consensus.cc:686] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 1 LEADER]: Becoming Leader. State: Replica: 63792240b63e4f4d86190d7d3d2dd3dc, State: Running, Role: LEADER
I20260809 08:42:24.203158 7115 consensus_queue.cc:237] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 } } peers { permanent_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37993 } } peers { permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 } }
I20260809 08:42:24.204035 6429 catalog_manager.cc:5495] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 63792240b63e4f4d86190d7d3d2dd3dc (127.31.63.67). New cstate: current_term: 1 leader_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63792240b63e4f4d86190d7d3d2dd3dc" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 } health_report { overall_health: UNKNOWN } } }
I20260809 08:42:24.214881 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.213529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53740 (local address 127.31.63.67:37993)
0809 08:42:24.213669 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.213672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.213683 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:42:24.213784 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.213785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.213805 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.213869 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.214581 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.214583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.214584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.214589 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.214609 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.214614 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.214676 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.214781 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.214796 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260809 08:42:24.216786 6643 raft_consensus.cc:1261] T 9b708a7eb2894e37864d1fd907904e82 P 6f714369b46c482d87533ee372f6b7a4 [term 1 FOLLOWER]: Refusing update from remote peer 63792240b63e4f4d86190d7d3d2dd3dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:24.216871 6835 raft_consensus.cc:1261] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23 [term 1 FOLLOWER]: Refusing update from remote peer 63792240b63e4f4d86190d7d3d2dd3dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:24.217201 7120 consensus_queue.cc:1035] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f714369b46c482d87533ee372f6b7a4" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 40939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:24.217361 7115 consensus_queue.cc:1035] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "90282f2dece84a7b8eaceb396ef16e23" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:24.221131 7130 mvcc.cc:205] Tried to move back new op lower bound from 7316541211509174272 to 7316541211456811008. Current Snapshot: MvccSnapshot[applied={T|T < 7316541211509174272 or (T in {7316541211509174272})}]
I20260809 08:42:24.222445 6430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:24.222549 6430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:24.223351 6430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb5ae6de700e46159222fe4398aeb987: Sending DeleteTablet for 3 replicas of tablet 9b708a7eb2894e37864d1fd907904e82
I20260809 08:42:24.223752 6800 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b708a7eb2894e37864d1fd907904e82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:24 UTC) from {username='slave'} at 127.0.0.1:55650
I20260809 08:42:24.223869 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 6517
I20260809 08:42:24.223892 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b708a7eb2894e37864d1fd907904e82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:24 UTC) from {username='slave'} at 127.0.0.1:53732
I20260809 08:42:24.237903 7138 tablet_replica.cc:330] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: stopping tablet replica
I20260809 08:42:24.238098 7138 raft_consensus.cc:2229] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:42:24.238343 7137 tablet_replica.cc:330] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23: stopping tablet replica
W20260809 08:42:24.238466 6407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:40939 (error 108)
I20260809 08:42:24.238621 7137 raft_consensus.cc:2229] T 9b708a7eb2894e37864d1fd907904e82 P 90282f2dece84a7b8eaceb396ef16e23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:24.238669 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 6712
I20260809 08:42:24.238895 7138 raft_consensus.cc:2258] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:24.239478 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.239343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:40939 (local address 127.0.0.1:38072)
0809 08:42:24.239402 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.239437 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:40939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 08:42:24.239581 6407 catalog_manager.cc:4584] TS 6f714369b46c482d87533ee372f6b7a4 (127.31.63.65:40939): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9b708a7eb2894e37864d1fd907904e82: Network error: Client connection negotiation failed: client connection to 127.31.63.65:40939: connect: Connection refused (error 111)
I20260809 08:42:24.239956 7138 ts_tablet_manager.cc:1905] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:24.241752 7138 ts_tablet_manager.cc:1918] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:42:24.241858 7138 log.cc:1196] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/9b708a7eb2894e37864d1fd907904e82
I20260809 08:42:24.242142 7138 ts_tablet_manager.cc:1939] T 9b708a7eb2894e37864d1fd907904e82 P 63792240b63e4f4d86190d7d3d2dd3dc: Deleting consensus metadata
I20260809 08:42:24.242586 6407 catalog_manager.cc:4841] TS 63792240b63e4f4d86190d7d3d2dd3dc (127.31.63.67:37993): tablet 9b708a7eb2894e37864d1fd907904e82 (table test-table [id=95505854e2a64fe696dd43a61afee5a9]) successfully deleted
I20260809 08:42:24.248384 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.248241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:46821 (local address 127.0.0.1:55660)
0809 08:42:24.248306 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.248344 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.66:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 08:42:24.248426 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 6907
I20260809 08:42:24.261464 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 6376
2026-08-09T08:42:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (804 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 08:42:24.289026 31997 mini_oidc.cc:152] Starting JWKS server
I20260809 08:42:24.290619 31997 webserver.cc:460] Webserver started at https://127.0.0.1:34263/ using document root <none> and password file <none>
I20260809 08:42:24.290755 31997 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 08:42:24.290980 31997 webserver.cc:460] Webserver started at http://127.0.0.1:41921/ using document root <none> and password file <none>
2026-08-09T08:42:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:24Z Disabled control of system clock
I20260809 08:42:24.300459 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37589
--webserver_interface=127.31.63.126
--webserver_port=0
--builtin_ntp_servers=127.31.63.84:42035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:37589
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34263/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/testchainca.cert with env {}
W20260809 08:42:24.387797 7146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:24.388059 7146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:24.388099 7146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:24.389955 7146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:24.390030 7146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:24.390053 7146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:24.390072 7146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:24.390100 7146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:42:24.390117 7146 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:42:24.390132 7146 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34263/jwks.json
I20260809 08:42:24.392844 7146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:37589
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34263/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:24.393126 7146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:24.393380 7146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:24.400158 7152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:24.400225 7153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:24.400158 7155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:24.400733 7146 server_base.cc:1034] running on GCE node
I20260809 08:42:24.400996 7146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:24.401328 7146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:24.402480 7146 hybrid_clock.cc:648] HybridClock initialized: now 1786264944402454 us; error 42 us; skew 500 ppm
I20260809 08:42:24.404362 7146 webserver.cc:460] Webserver started at http://127.31.63.126:42477/ using document root <none> and password file <none>
I20260809 08:42:24.404662 7146 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:24.404726 7146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:24.404867 7146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:24.406107 7146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd"
format_stamp: "Formatted at 2026-08-09 08:42:24 on dist-test-slave-gkw7"
server_key: "648504dc7ddb9fae05800101e2deea1f"
server_key_iv: "132c4284aaf6d18606e0ccaf6d6a9fc4"
server_key_version: "encryptionkey@0"
I20260809 08:42:24.406497 7146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd"
format_stamp: "Formatted at 2026-08-09 08:42:24 on dist-test-slave-gkw7"
server_key: "648504dc7ddb9fae05800101e2deea1f"
server_key_iv: "132c4284aaf6d18606e0ccaf6d6a9fc4"
server_key_version: "encryptionkey@0"
I20260809 08:42:24.409379 7146 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 08:42:24.411996 7162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:24.412195 7146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:24.412307 7146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd"
format_stamp: "Formatted at 2026-08-09 08:42:24 on dist-test-slave-gkw7"
server_key: "648504dc7ddb9fae05800101e2deea1f"
server_key_iv: "132c4284aaf6d18606e0ccaf6d6a9fc4"
server_key_version: "encryptionkey@0"
I20260809 08:42:24.412391 7146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:24.425594 7146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:24.425948 7146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:24.426095 7146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:24.431249 7146 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:37589
I20260809 08:42:24.431294 7241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:37589 every 8 connection(s)
I20260809 08:42:24.431692 7146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:24.433053 7242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:24.435227 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 7146
I20260809 08:42:24.435339 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:24.435534 31997 external_mini_cluster.cc:1428] Setting key 4eaf2ef657f1b5842faa2b2bc8f4c035
I20260809 08:42:24.436970 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd: Bootstrap starting.
I20260809 08:42:24.437798 7242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:24.438136 7242 log.cc:824] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:24.438812 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd: No bootstrap required, opened a new log
I20260809 08:42:24.440544 7242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37589 } }
I20260809 08:42:24.440702 7242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:24.440750 7242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f7c28ab1b5e4a75a4844fe38702c2fd, State: Initialized, Role: FOLLOWER
I20260809 08:42:24.440900 7242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37589 } }
I20260809 08:42:24.440980 7242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:24.441022 7242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:24.441080 7242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:24.441460 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.436272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.31.63.126:37589)
0809 08:42:24.436492 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.436499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.436527 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:42:24.436901 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.436907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.436966 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.437172 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.437396 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.438538 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.439423 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.439860 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.440001 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:24.440732 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.440736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.440737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.440945 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.441010 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.441026 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.441086 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.441161 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.441249 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260809 08:42:24.441682 7242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37589 } }
I20260809 08:42:24.441814 7242 leader_election.cc:302] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f7c28ab1b5e4a75a4844fe38702c2fd; no voters:
I20260809 08:42:24.441992 7242 leader_election.cc:288] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:24.442028 7247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:24.442178 7247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [term 1 LEADER]: Becoming Leader. State: Replica: 7f7c28ab1b5e4a75a4844fe38702c2fd, State: Running, Role: LEADER
I20260809 08:42:24.442270 7247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37589 } }
I20260809 08:42:24.442301 7242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:24.442420 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:42:24.442570 7247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f7c28ab1b5e4a75a4844fe38702c2fd. Latest consensus state: current_term: 1 leader_uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37589 } } }
I20260809 08:42:24.442663 7247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:24.442580 7251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7c28ab1b5e4a75a4844fe38702c2fd" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37589 } } }
I20260809 08:42:24.442824 7251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:24.442945 7257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:24.443464 7257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:24.445158 7257 catalog_manager.cc:1347] Generated new cluster ID: 800568bd65784e238ad0015515b55c12
I20260809 08:42:24.445214 7257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:24.458621 7257 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:24.459154 7257 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:24.461072 7257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f7c28ab1b5e4a75a4844fe38702c2fd: Generated new TSK 0
I20260809 08:42:24.461278 7257 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 08:42:24.489166 7292 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 08:42:24.489359 7292 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 08:42:24.489645 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.474012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.31.63.126:37589)
0809 08:42:24.474064 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.474066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.474205 (+ 139us) server_negotiation.cc:407] Connection header received
0809 08:42:24.474297 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.474298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.474308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.474366 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:42:24.474499 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.475009 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.476139 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.476348 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.476402 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:24.476632 (+ 230us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:42:24.489296 (+ 12664us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 08:42:24.489507 (+ 211us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55214: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 08:42:24.489737 7245 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55214: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 08:42:24.490192 7291 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:42:24.473907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37589 (local address 127.0.0.1:55214)
0809 08:42:24.474119 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.474131 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:24.474205 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:24.474371 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:24.474372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:24.474374 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:42:24.474441 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:24.474443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.475088 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.475089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:24.476052 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:24.476055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.476436 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.476437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:24.476505 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:24.476527 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:42:24.489609 (+ 13082us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 08:42:24.489765 (+ 156us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.63.126:37589: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":110,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.63.126:37589: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 08:42:24.490682 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 7146
2026-08-09T08:42:24Z chronyd exiting
I20260809 08:42:24.534897 31997 test_util.cc:175] -----------------------------------------------
I20260809 08:42:24.535000 31997 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786264918773806-31997-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (247 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 08:42:24.535776 31997 mini_oidc.cc:152] Starting JWKS server
I20260809 08:42:24.536492 31997 webserver.cc:460] Webserver started at https://127.0.0.1:37573/ using document root <none> and password file <none>
I20260809 08:42:24.536561 31997 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 08:42:24.536748 31997 webserver.cc:460] Webserver started at http://127.0.0.1:46835/ using document root <none> and password file <none>
2026-08-09T08:42:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:24Z Disabled control of system clock
I20260809 08:42:24.546090 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:36417
--webserver_interface=127.31.63.126
--webserver_port=0
--builtin_ntp_servers=127.31.63.84:36933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:36417
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37573/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/testchainca.cert with env {}
W20260809 08:42:24.631330 7300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:24.631531 7300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:24.631552 7300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:24.632723 7300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:24.632771 7300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:24.632783 7300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:24.632797 7300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:24.632808 7300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:42:24.632822 7300 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:42:24.632833 7300 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37573/jwks.json
I20260809 08:42:24.634402 7300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:36933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:36417
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:36417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37573/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:24.634898 7300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:24.635165 7300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:24.640971 7316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:24.641124 7300 server_base.cc:1034] running on GCE node
W20260809 08:42:24.640967 7312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:24.641230 7313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:24.641453 7300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:24.641848 7300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:24.643023 7300 hybrid_clock.cc:648] HybridClock initialized: now 1786264944643002 us; error 33 us; skew 500 ppm
I20260809 08:42:24.644706 7300 webserver.cc:460] Webserver started at http://127.31.63.126:33787/ using document root <none> and password file <none>
I20260809 08:42:24.644989 7300 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:24.645117 7300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:24.645293 7300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:24.646632 7300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "3ca211a8ee6947fc84e3e69556d5a0fe"
format_stamp: "Formatted at 2026-08-09 08:42:24 on dist-test-slave-gkw7"
server_key: "3048ce9cb0c1cee860dd91b05d652dae"
server_key_iv: "06bd3ce7bd023f0891ff3b9586008615"
server_key_version: "encryptionkey@0"
I20260809 08:42:24.647073 7300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "3ca211a8ee6947fc84e3e69556d5a0fe"
format_stamp: "Formatted at 2026-08-09 08:42:24 on dist-test-slave-gkw7"
server_key: "3048ce9cb0c1cee860dd91b05d652dae"
server_key_iv: "06bd3ce7bd023f0891ff3b9586008615"
server_key_version: "encryptionkey@0"
I20260809 08:42:24.648614 7300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:24.649417 7323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:24.649624 7300 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:24.649739 7300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "3ca211a8ee6947fc84e3e69556d5a0fe"
format_stamp: "Formatted at 2026-08-09 08:42:24 on dist-test-slave-gkw7"
server_key: "3048ce9cb0c1cee860dd91b05d652dae"
server_key_iv: "06bd3ce7bd023f0891ff3b9586008615"
server_key_version: "encryptionkey@0"
I20260809 08:42:24.649819 7300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:24.678920 7300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:24.679333 7300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:24.679505 7300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:24.683702 7300 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:36417
I20260809 08:42:24.683748 7401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:36417 every 8 connection(s)
I20260809 08:42:24.684142 7300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:24.684825 7402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:24.687631 7402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe: Bootstrap starting.
I20260809 08:42:24.688185 7402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:24.688453 7402 log.cc:824] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:24.689162 7402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe: No bootstrap required, opened a new log
I20260809 08:42:24.690821 7402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca211a8ee6947fc84e3e69556d5a0fe" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36417 } }
I20260809 08:42:24.690964 7402 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:24.690986 7402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ca211a8ee6947fc84e3e69556d5a0fe, State: Initialized, Role: FOLLOWER
I20260809 08:42:24.691005 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 7300
I20260809 08:42:24.691128 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:24.691169 7402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca211a8ee6947fc84e3e69556d5a0fe" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36417 } }
I20260809 08:42:24.691270 7402 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:24.691320 7402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:24.691371 7402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:24.691385 31997 external_mini_cluster.cc:1428] Setting key 1a62e4b69aebe4c24af7bb9a774f0784
I20260809 08:42:24.692204 7402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca211a8ee6947fc84e3e69556d5a0fe" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36417 } }
I20260809 08:42:24.692332 7402 leader_election.cc:302] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ca211a8ee6947fc84e3e69556d5a0fe; no voters:
I20260809 08:42:24.692492 7402 leader_election.cc:288] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:24.692550 7410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:24.692704 7410 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [term 1 LEADER]: Becoming Leader. State: Replica: 3ca211a8ee6947fc84e3e69556d5a0fe, State: Running, Role: LEADER
I20260809 08:42:24.692776 7402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:24.692803 7410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca211a8ee6947fc84e3e69556d5a0fe" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36417 } }
I20260809 08:42:24.693130 7410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ca211a8ee6947fc84e3e69556d5a0fe. Latest consensus state: current_term: 1 leader_uuid: "3ca211a8ee6947fc84e3e69556d5a0fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca211a8ee6947fc84e3e69556d5a0fe" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36417 } } }
I20260809 08:42:24.693185 7410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:24.693435 7412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ca211a8ee6947fc84e3e69556d5a0fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca211a8ee6947fc84e3e69556d5a0fe" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36417 } } }
I20260809 08:42:24.693509 7412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:24.694483 7414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:24.695120 7414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:24.696733 7414 catalog_manager.cc:1347] Generated new cluster ID: 23acc953ee5f4a21af2438ddeab0b66d
I20260809 08:42:24.696797 7414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:24.702695 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.691919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35848 (local address 127.31.63.126:36417)
0809 08:42:24.692186 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.692196 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.692236 (+ 40us) server_negotiation.cc:407] Connection header received
0809 08:42:24.692357 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.692365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.692459 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.692925 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.693034 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.694821 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.695958 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.696541 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.701634 (+ 5093us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:24.701833 (+ 199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.701836 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.701836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.702041 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.702079 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.702092 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.702159 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.702279 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.702467 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":175,"thread_start_us":110,"threads_started":1}
I20260809 08:42:24.703655 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:42:24.703956 7414 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:24.704650 7414 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:24.708740 7414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ca211a8ee6947fc84e3e69556d5a0fe: Generated new TSK 0
I20260809 08:42:24.708920 7414 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 08:42:24.728017 7443 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 08:42:24.728101 7443 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 08:42:24.728502 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.713784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35864 (local address 127.31.63.126:36417)
0809 08:42:24.713850 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.713852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.713973 (+ 121us) server_negotiation.cc:407] Connection header received
0809 08:42:24.714051 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.714052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.714065 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.714124 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:42:24.714254 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.714753 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.715579 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.715816 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.715894 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:24.716080 (+ 186us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:42:24.728248 (+ 12168us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 08:42:24.728417 (+ 169us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35864: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 08:42:24.728566 7409 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35864: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 08:42:24.728593 7442 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:42:24.713712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:36417 (local address 127.0.0.1:35864)
0809 08:42:24.713900 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.713910 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:24.713990 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:24.714132 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:24.714133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:24.714135 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:42:24.714189 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:24.714191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.714860 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.714861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:24.715477 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:24.715478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.715898 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.715900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:24.715960 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:24.715964 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:42:24.728360 (+ 12396us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 08:42:24.728487 (+ 127us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.63.126:36417: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37573/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":109,"threads_started":1}
I20260809 08:42:24.728828 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 7300
2026-08-09T08:42:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (241 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 08:42:24.777022 31997 mini_oidc.cc:152] Starting JWKS server
I20260809 08:42:24.777767 31997 webserver.cc:460] Webserver started at https://127.0.0.1:44497/ using document root <none> and password file <none>
I20260809 08:42:24.777812 31997 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 08:42:24.777977 31997 webserver.cc:460] Webserver started at http://127.0.0.1:37247/ using document root <none> and password file <none>
2026-08-09T08:42:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:24Z Disabled control of system clock
I20260809 08:42:24.787312 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:39391
--webserver_interface=127.31.63.126
--webserver_port=0
--builtin_ntp_servers=127.31.63.84:35745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:39391
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44497/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 08:42:24.870095 7450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:24.870303 7450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:24.870324 7450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:24.871570 7450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:24.871613 7450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:24.871625 7450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:24.871637 7450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:24.871650 7450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:42:24.871695 7450 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:42:24.871711 7450 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44497/jwks.json
I20260809 08:42:24.873098 7450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:39391
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:39391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44497/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:24.873353 7450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:24.873699 7450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:24.878691 7457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:24.878721 7456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:24.878851 7460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:24.878935 7450 server_base.cc:1034] running on GCE node
I20260809 08:42:24.879186 7450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:24.879529 7450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:24.880721 7450 hybrid_clock.cc:648] HybridClock initialized: now 1786264944880706 us; error 33 us; skew 500 ppm
I20260809 08:42:24.882102 7450 webserver.cc:460] Webserver started at http://127.31.63.126:46809/ using document root <none> and password file <none>
I20260809 08:42:24.882376 7450 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:24.882452 7450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:24.882606 7450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:24.883682 7450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "348e88d1b03c4241a35edd336c34502e"
format_stamp: "Formatted at 2026-08-09 08:42:24 on dist-test-slave-gkw7"
server_key: "6508f4688eeaa1765ae9843adbd0f9be"
server_key_iv: "69b652d2dbdec553b656e2a151b1d7fc"
server_key_version: "encryptionkey@0"
I20260809 08:42:24.884126 7450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "348e88d1b03c4241a35edd336c34502e"
format_stamp: "Formatted at 2026-08-09 08:42:24 on dist-test-slave-gkw7"
server_key: "6508f4688eeaa1765ae9843adbd0f9be"
server_key_iv: "69b652d2dbdec553b656e2a151b1d7fc"
server_key_version: "encryptionkey@0"
I20260809 08:42:24.885454 7450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:42:24.886248 7468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:24.886468 7450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:24.886606 7450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "348e88d1b03c4241a35edd336c34502e"
format_stamp: "Formatted at 2026-08-09 08:42:24 on dist-test-slave-gkw7"
server_key: "6508f4688eeaa1765ae9843adbd0f9be"
server_key_iv: "69b652d2dbdec553b656e2a151b1d7fc"
server_key_version: "encryptionkey@0"
I20260809 08:42:24.886691 7450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:24.912210 7450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:24.912627 7450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:24.912825 7450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:24.917301 7450 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:39391
I20260809 08:42:24.917380 7544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:39391 every 8 connection(s)
I20260809 08:42:24.917817 7450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:24.918593 7546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:24.921360 7546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e: Bootstrap starting.
I20260809 08:42:24.922047 7546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:24.922271 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 7450
I20260809 08:42:24.922411 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:24.922425 7546 log.cc:824] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:24.922631 31997 external_mini_cluster.cc:1428] Setting key 4f22de42a4c08b5c70c3ae10f1fad394
I20260809 08:42:24.923281 7546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e: No bootstrap required, opened a new log
I20260809 08:42:24.925052 7546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348e88d1b03c4241a35edd336c34502e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39391 } }
I20260809 08:42:24.925196 7546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:24.925227 7546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348e88d1b03c4241a35edd336c34502e, State: Initialized, Role: FOLLOWER
I20260809 08:42:24.925400 7546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348e88d1b03c4241a35edd336c34502e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39391 } }
I20260809 08:42:24.925498 7546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:24.925581 7546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:24.925642 7546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:24.926239 7546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348e88d1b03c4241a35edd336c34502e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39391 } }
I20260809 08:42:24.926369 7546 leader_election.cc:302] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 348e88d1b03c4241a35edd336c34502e; no voters:
I20260809 08:42:24.926555 7546 leader_election.cc:288] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:24.926604 7553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:24.926755 7553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [term 1 LEADER]: Becoming Leader. State: Replica: 348e88d1b03c4241a35edd336c34502e, State: Running, Role: LEADER
I20260809 08:42:24.926836 7546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:24.926842 7553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348e88d1b03c4241a35edd336c34502e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39391 } }
I20260809 08:42:24.927171 7553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 348e88d1b03c4241a35edd336c34502e. Latest consensus state: current_term: 1 leader_uuid: "348e88d1b03c4241a35edd336c34502e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348e88d1b03c4241a35edd336c34502e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39391 } } }
I20260809 08:42:24.927234 7553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:24.927367 7555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "348e88d1b03c4241a35edd336c34502e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348e88d1b03c4241a35edd336c34502e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39391 } } }
I20260809 08:42:24.927493 7555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:24.928692 7559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:24.929210 7559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:24.930968 7559 catalog_manager.cc:1347] Generated new cluster ID: e856ba25c0d843d194a891d5bec176c6
I20260809 08:42:24.931025 7559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:24.931066 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.923196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.31.63.126:39391)
0809 08:42:24.923462 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.923471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.923500 (+ 29us) server_negotiation.cc:407] Connection header received
0809 08:42:24.925049 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.925059 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.925137 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.925327 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:24.925620 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.927396 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.928648 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.929236 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.929982 (+ 746us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:24.930296 (+ 314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:24.930299 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:24.930299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:24.930526 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:24.930567 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:24.930581 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:24.930678 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:24.930758 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 08:42:24.930837 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":103,"threads_started":1}
I20260809 08:42:24.932010 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:42:24.936260 7559 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:24.936736 7559 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:24.943053 7559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 348e88d1b03c4241a35edd336c34502e: Generated new TSK 0
I20260809 08:42:24.943243 7559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:42:24.952600 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:24.949055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52164 (local address 127.31.63.126:39391)
0809 08:42:24.949126 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:24.949129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:24.949224 (+ 95us) server_negotiation.cc:407] Connection header received
0809 08:42:24.949301 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:24.949302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:24.949315 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:24.949385 (+ 70us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:42:24.949510 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.950002 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.950828 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.951033 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.951116 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:24.951301 (+ 185us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:42:24.952383 (+ 1082us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44497/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 08:42:24.952511 (+ 128us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52164: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44497/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 08:42:24.952657 7552 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52164: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44497/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 08:42:24.952955 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:42:24.949003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:39391 (local address 127.0.0.1:52164)
0809 08:42:24.949156 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:24.949165 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:24.949229 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:24.949385 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:24.949386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:24.949388 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:42:24.949446 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:24.949448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.950107 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.950108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:24.950735 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:24.950737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:24.951115 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:24.951116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:24.951174 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:24.951179 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:42:24.952761 (+ 1582us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44497/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 08:42:24.952862 (+ 101us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.63.126:39391: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44497/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260809 08:42:24.953269 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 7450
E20260809 08:42:24.978192 7593 webserver.cc:533] Webserver: sslize failed:
E20260809 08:42:24.978304 7593 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T08:42:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T08:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:25Z Disabled control of system clock
I20260809 08:42:25.015071 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:39055
--webserver_interface=127.31.63.126
--webserver_port=0
--builtin_ntp_servers=127.31.63.84:34909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:39055
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/testchainca.cert with env {}
W20260809 08:42:25.098541 7597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:25.098757 7597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:25.098781 7597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:25.099970 7597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:25.100014 7597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:25.100028 7597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:25.100068 7597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:25.100098 7597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:42:25.100116 7597 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:42:25.100138 7597 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 08:42:25.101663 7597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:39055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:39055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:25.101940 7597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:25.102167 7597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:25.107240 7603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:25.107244 7602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:25.107223 7606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:25.107520 7597 server_base.cc:1034] running on GCE node
I20260809 08:42:25.107820 7597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:25.108134 7597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:25.109309 7597 hybrid_clock.cc:648] HybridClock initialized: now 1786264945109292 us; error 43 us; skew 500 ppm
I20260809 08:42:25.110699 7597 webserver.cc:460] Webserver started at http://127.31.63.126:46095/ using document root <none> and password file <none>
I20260809 08:42:25.110987 7597 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:25.111061 7597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:25.111224 7597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:25.112279 7597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "13acad8d32a649c38cd8c5f8f37af0e8"
format_stamp: "Formatted at 2026-08-09 08:42:25 on dist-test-slave-gkw7"
server_key: "d8ab0536fcb32d8b2ca2b69ec1728d30"
server_key_iv: "3fabf2d05cc82437553a7786058894d4"
server_key_version: "encryptionkey@0"
I20260809 08:42:25.112643 7597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "13acad8d32a649c38cd8c5f8f37af0e8"
format_stamp: "Formatted at 2026-08-09 08:42:25 on dist-test-slave-gkw7"
server_key: "d8ab0536fcb32d8b2ca2b69ec1728d30"
server_key_iv: "3fabf2d05cc82437553a7786058894d4"
server_key_version: "encryptionkey@0"
I20260809 08:42:25.114089 7597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 08:42:25.114816 7615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:25.114984 7597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:25.115093 7597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "13acad8d32a649c38cd8c5f8f37af0e8"
format_stamp: "Formatted at 2026-08-09 08:42:25 on dist-test-slave-gkw7"
server_key: "d8ab0536fcb32d8b2ca2b69ec1728d30"
server_key_iv: "3fabf2d05cc82437553a7786058894d4"
server_key_version: "encryptionkey@0"
I20260809 08:42:25.115170 7597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:25.127907 7597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:25.128245 7597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:25.128417 7597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:25.132746 7597 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:39055
I20260809 08:42:25.132772 7694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:39055 every 8 connection(s)
I20260809 08:42:25.133138 7597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:25.134212 7695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:25.136477 7695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8: Bootstrap starting.
I20260809 08:42:25.137027 7695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:25.137272 7695 log.cc:824] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:25.137992 7695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8: No bootstrap required, opened a new log
I20260809 08:42:25.139578 7695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13acad8d32a649c38cd8c5f8f37af0e8" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39055 } }
I20260809 08:42:25.139724 7695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:25.139709 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 7597
I20260809 08:42:25.139748 7695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13acad8d32a649c38cd8c5f8f37af0e8, State: Initialized, Role: FOLLOWER
I20260809 08:42:25.139860 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:25.139925 7695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13acad8d32a649c38cd8c5f8f37af0e8" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39055 } }
I20260809 08:42:25.140022 7695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:25.140067 7695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:25.140092 31997 external_mini_cluster.cc:1428] Setting key f2812f1cd69907a106889cb4eb58a71a
I20260809 08:42:25.140116 7695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:25.140915 7695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13acad8d32a649c38cd8c5f8f37af0e8" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39055 } }
I20260809 08:42:25.141027 7695 leader_election.cc:302] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13acad8d32a649c38cd8c5f8f37af0e8; no voters:
I20260809 08:42:25.141203 7695 leader_election.cc:288] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:25.141265 7704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:25.141410 7704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [term 1 LEADER]: Becoming Leader. State: Replica: 13acad8d32a649c38cd8c5f8f37af0e8, State: Running, Role: LEADER
I20260809 08:42:25.141481 7695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:25.141507 7704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13acad8d32a649c38cd8c5f8f37af0e8" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39055 } }
I20260809 08:42:25.141877 7704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13acad8d32a649c38cd8c5f8f37af0e8. Latest consensus state: current_term: 1 leader_uuid: "13acad8d32a649c38cd8c5f8f37af0e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13acad8d32a649c38cd8c5f8f37af0e8" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39055 } } }
I20260809 08:42:25.141969 7704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:25.141887 7707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13acad8d32a649c38cd8c5f8f37af0e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13acad8d32a649c38cd8c5f8f37af0e8" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39055 } } }
I20260809 08:42:25.142099 7707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:25.142181 7712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:25.142709 7712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:25.144507 7712 catalog_manager.cc:1347] Generated new cluster ID: 7cb421bb0bf44cce9bc739cd3dbd0f0a
I20260809 08:42:25.144564 7712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:42:25.146942 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:25.140639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.31.63.126:39055)
0809 08:42:25.140905 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:25.140915 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:25.140951 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:42:25.141070 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:25.141078 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:25.141155 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:25.141314 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:25.141723 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:25.143604 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:25.144716 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:25.145353 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:25.145482 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:25.146208 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:25.146212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:25.146213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:42:25.146452 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:25.146493 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:42:25.146509 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:25.146583 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:25.146648 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 08:42:25.146732 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
I20260809 08:42:25.147868 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:42:25.152819 7712 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:25.153564 7712 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:25.159317 7712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13acad8d32a649c38cd8c5f8f37af0e8: Generated new TSK 0
I20260809 08:42:25.159513 7712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:42:25.650583 7703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:25.166619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55546 (local address 127.31.63.126:39055)
0809 08:42:25.166680 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:25.166682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:25.166776 (+ 94us) server_negotiation.cc:407] Connection header received
0809 08:42:25.166829 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:25.166830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:25.166844 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:25.166925 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:42:25.167023 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:25.167801 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:25.168579 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:25.168912 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:25.168986 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:25.169130 (+ 144us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:42:25.650339 (+481209us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 08:42:25.650517 (+ 178us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55546: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 08:42:25.650648 7703 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55546: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 08:42:25.650760 7735 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:42:25.166520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:39055 (local address 127.0.0.1:55546)
0809 08:42:25.166711 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:25.166721 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:25.166774 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:25.166929 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:25.166929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:25.166931 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:42:25.166981 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:25.166982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:25.167891 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:25.167891 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:25.168499 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:25.168500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:25.168995 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:25.168996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:25.169037 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:25.169042 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:42:25.650487 (+481445us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 08:42:25.650615 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.63.126:39055: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":114,"threads_started":1}
I20260809 08:42:25.651180 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 7597
2026-08-09T08:42:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (670 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:42:25 dist-test-slave-gkw7 krb5kdc[7755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:42:25 dist-test-slave-gkw7 krb5kdc[7755](info): set up 2 sockets
Aug 09 08:42:25 dist-test-slave-gkw7 krb5kdc[7755](info): commencing operation
krb5kdc: starting...
W20260809 08:42:27.060256 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:42:27.185688 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:32787
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:45543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:32787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:27.268734 7773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:27.268958 7773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:27.268982 7773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:27.270177 7773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:27.270217 7773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:27.270232 7773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:27.270244 7773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:27.270256 7773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:27.271768 7773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:45543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:32787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:32787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:27.271978 7773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:27.272151 7773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:27.279031 7784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:27.279038 7781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:27.279089 7782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:27.279449 7773 server_base.cc:1034] running on GCE node
I20260809 08:42:27.279735 7773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:27.280083 7773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:27.281255 7773 hybrid_clock.cc:648] HybridClock initialized: now 1786264947281215 us; error 62 us; skew 500 ppm
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.287503 7773 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:42:27.287959 7773 webserver.cc:460] Webserver started at http://127.31.63.126:41379/ using document root <none> and password file <none>
I20260809 08:42:27.288210 7773 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:27.288290 7773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:27.288446 7773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:27.289585 7773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "e51674bd83494af2bbf055a7fefad581"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "4bc614e97a14463771ed067b99535816"
server_key_iv: "f19a024b4a734f0e9fc591819a19d145"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.289964 7773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "e51674bd83494af2bbf055a7fefad581"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "4bc614e97a14463771ed067b99535816"
server_key_iv: "f19a024b4a734f0e9fc591819a19d145"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.291419 7773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:42:27.292151 7793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:27.292351 7773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:42:27.292462 7773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "e51674bd83494af2bbf055a7fefad581"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "4bc614e97a14463771ed067b99535816"
server_key_iv: "f19a024b4a734f0e9fc591819a19d145"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.292537 7773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:27.309470 7773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:27.311972 7773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:27.312142 7773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:27.316476 7773 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:32787
I20260809 08:42:27.316511 7875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:32787 every 8 connection(s)
I20260809 08:42:27.316890 7773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:27.317852 7876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:27.320161 7876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581: Bootstrap starting.
I20260809 08:42:27.320776 7876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:27.321089 7876 log.cc:824] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:27.321183 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 7773
I20260809 08:42:27.321408 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:27.321702 31997 external_mini_cluster.cc:1428] Setting key 61ec3ec3503e6c1d5bc72c51b379723c
I20260809 08:42:27.322129 7876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581: No bootstrap required, opened a new log
I20260809 08:42:27.324302 7876 raft_consensus.cc:348] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51674bd83494af2bbf055a7fefad581" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32787 } }
I20260809 08:42:27.324482 7876 raft_consensus.cc:374] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:27.324515 7876 raft_consensus.cc:729] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51674bd83494af2bbf055a7fefad581, State: Initialized, Role: FOLLOWER
I20260809 08:42:27.324672 7876 consensus_queue.cc:260] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51674bd83494af2bbf055a7fefad581" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32787 } }
I20260809 08:42:27.324764 7876 raft_consensus.cc:388] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:27.324796 7876 raft_consensus.cc:482] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:27.324841 7876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:27.325551 7876 raft_consensus.cc:504] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51674bd83494af2bbf055a7fefad581" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32787 } }
I20260809 08:42:27.325656 7876 leader_election.cc:302] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e51674bd83494af2bbf055a7fefad581; no voters:
I20260809 08:42:27.325799 7876 leader_election.cc:288] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:27.325865 7886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:27.326022 7876 sys_catalog.cc:564] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:27.326031 7886 raft_consensus.cc:686] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [term 1 LEADER]: Becoming Leader. State: Replica: e51674bd83494af2bbf055a7fefad581, State: Running, Role: LEADER
I20260809 08:42:27.326150 7886 consensus_queue.cc:237] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51674bd83494af2bbf055a7fefad581" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32787 } }
I20260809 08:42:27.326475 7886 sys_catalog.cc:455] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e51674bd83494af2bbf055a7fefad581. Latest consensus state: current_term: 1 leader_uuid: "e51674bd83494af2bbf055a7fefad581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51674bd83494af2bbf055a7fefad581" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32787 } } }
I20260809 08:42:27.326571 7886 sys_catalog.cc:458] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:27.326488 7887 sys_catalog.cc:455] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e51674bd83494af2bbf055a7fefad581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51674bd83494af2bbf055a7fefad581" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32787 } } }
I20260809 08:42:27.326705 7887 sys_catalog.cc:458] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:27.326821 7896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:27.327384 7896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:27.329587 7896 catalog_manager.cc:1347] Generated new cluster ID: d5bc06f3d27b4006806b241ae901ccda
I20260809 08:42:27.329643 7896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:27.335211 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:27.322300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56178 (local address 127.31.63.126:32787)
0809 08:42:27.322526 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:27.322536 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:27.322578 (+ 42us) server_negotiation.cc:407] Connection header received
0809 08:42:27.323485 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:27.323495 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:27.323559 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:27.323728 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:27.324334 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.326057 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.327295 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.327936 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.328354 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:27.330928 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:27.330932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:27.330932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:27.330945 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:27.332971 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.333377 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.333381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.333381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.333497 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.333733 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.333734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.333734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.334072 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:27.334643 (+ 571us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:27.334729 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 08:42:27.334999 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":123,"threads_started":1}
I20260809 08:42:27.362354 7896 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:27.362933 7896 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:27.366237 7896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e51674bd83494af2bbf055a7fefad581: Generated new TSK 0
I20260809 08:42:27.366451 7896 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:42:27.394555 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:32787
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:45543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:27.478451 7912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:27.478696 7912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:27.478745 7912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:27.479945 7912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:27.480015 7912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:27.480062 7912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:27.481644 7912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:45543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:32787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:27.481946 7912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:27.482187 7912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:27.487511 7917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:27.487540 7920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:27.487675 7918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:27.487768 7912 server_base.cc:1034] running on GCE node
I20260809 08:42:27.487977 7912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:27.488234 7912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:27.489405 7912 hybrid_clock.cc:648] HybridClock initialized: now 1786264947489392 us; error 30 us; skew 500 ppm
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.492056 7912 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:42:27.492530 7912 webserver.cc:460] Webserver started at http://127.31.63.65:36571/ using document root <none> and password file <none>
I20260809 08:42:27.492725 7912 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:27.492769 7912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:27.492859 7912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:27.493958 7912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "63e2935e7d8b46998c913e452b63a494"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "6c92514c02ae199d993bb9b2244a4e00"
server_key_iv: "6108a042b13eef01dddfee5556ea5f78"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.494266 7912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "63e2935e7d8b46998c913e452b63a494"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "6c92514c02ae199d993bb9b2244a4e00"
server_key_iv: "6108a042b13eef01dddfee5556ea5f78"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.495776 7912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:42:27.496594 7929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:27.496814 7912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:27.496891 7912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "63e2935e7d8b46998c913e452b63a494"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "6c92514c02ae199d993bb9b2244a4e00"
server_key_iv: "6108a042b13eef01dddfee5556ea5f78"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.496946 7912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:27.513512 7912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:27.515923 7912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:27.516037 7912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:27.516234 7912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:27.516589 7912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:27.516624 7912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:27.516669 7912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:27.516695 7912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:27.523336 7912 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:33467
I20260809 08:42:27.523398 8093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:33467 every 8 connection(s)
I20260809 08:42:27.523698 7912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:27.530778 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 7912
I20260809 08:42:27.530907 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:27.531117 31997 external_mini_cluster.cc:1428] Setting key 46b87b66288433b7b31193980e60642a
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:27.535521 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:27.524504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:56579 (local address 127.31.63.126:32787)
0809 08:42:27.524598 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:27.524600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:27.524772 (+ 172us) server_negotiation.cc:407] Connection header received
0809 08:42:27.525852 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:27.525853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:27.525867 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:27.525942 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:27.526663 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.527292 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.528998 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.529334 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.529413 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:27.532050 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:27.532051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:27.532052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:27.532061 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:27.533749 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.534418 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.534419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.534420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.534480 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.534779 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.534780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.534780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.534888 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:27.534940 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:27.535329 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 08:42:27.535444 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 08:42:27.535727 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:27.524438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:32787 (local address 127.31.63.65:56579)
0809 08:42:27.524633 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:27.524678 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:27.525731 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:27.526026 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:27.526030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:27.526290 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:27.526484 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:27.526489 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.527402 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.527404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:27.528889 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:27.528893 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.529443 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.529444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:27.529618 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:27.530164 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:27.530202 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:27.531966 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:27.533853 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:27.533863 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:27.533866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:27.534227 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:27.534557 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:27.534558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:27.534559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:27.534708 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:27.534957 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:27.534965 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:27.535145 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 08:42:27.535272 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 08:42:27.535457 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260809 08:42:27.536300 8095 heartbeater.cc:347] Connected to a master server at 127.31.63.126:32787
I20260809 08:42:27.536398 8095 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:27.536607 8095 heartbeater.cc:511] Master 127.31.63.126:32787 requested a full tablet report, sending...
I20260809 08:42:27.537207 7816 ts_manager.cc:194] Registered new tserver with Master: 63e2935e7d8b46998c913e452b63a494 (127.31.63.65:33467)
I20260809 08:42:27.537873 7816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:56579
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:42:27.558861 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:32787
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:45543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:27.641566 8100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:27.641857 8100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:27.641901 8100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:27.643488 8100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:27.643586 8100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:27.643638 8100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:27.645275 8100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:45543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:32787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:27.645613 8100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:27.645877 8100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:27.651356 8108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:27.651268 8113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:27.651268 8106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:27.651705 8100 server_base.cc:1034] running on GCE node
I20260809 08:42:27.651916 8100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:27.652127 8100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:27.653290 8100 hybrid_clock.cc:648] HybridClock initialized: now 1786264947653274 us; error 50 us; skew 500 ppm
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.655957 8100 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:42:27.656486 8100 webserver.cc:460] Webserver started at http://127.31.63.66:34953/ using document root <none> and password file <none>
I20260809 08:42:27.656742 8100 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:27.656816 8100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:27.656972 8100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:27.658142 8100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "aa295c6ab4a94b9abf33b060c7d523e7"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "541780005b439b02ff34b97a7a99cf54"
server_key_iv: "037efc6bbc40362d49fbb67b65fce331"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.658525 8100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "aa295c6ab4a94b9abf33b060c7d523e7"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "541780005b439b02ff34b97a7a99cf54"
server_key_iv: "037efc6bbc40362d49fbb67b65fce331"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.659955 8100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:42:27.660792 8120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:27.660986 8100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:27.661096 8100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "aa295c6ab4a94b9abf33b060c7d523e7"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "541780005b439b02ff34b97a7a99cf54"
server_key_iv: "037efc6bbc40362d49fbb67b65fce331"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.661199 8100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:27.668907 8100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:27.671164 8100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:27.671342 8100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:27.671591 8100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:27.671931 8100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:27.671998 8100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:27.672041 8100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:27.672066 8100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:27.678576 8100 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:46301
I20260809 08:42:27.678632 8281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:46301 every 8 connection(s)
I20260809 08:42:27.678958 8100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:42:27.684296 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 8100
I20260809 08:42:27.684394 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:42:27.684633 31997 external_mini_cluster.cc:1428] Setting key 7e3daa2a7169b128d51e935050b3e57e
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:27.689232 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:27.679727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:47429 (local address 127.31.63.126:32787)
0809 08:42:27.679819 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:27.679822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:27.680031 (+ 209us) server_negotiation.cc:407] Connection header received
0809 08:42:27.681012 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:27.681013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:27.681023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:27.681079 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:27.681776 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.682283 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.683247 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.683431 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.683482 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:27.686094 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:27.686101 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:27.686101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:27.686110 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:27.687695 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.688201 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.688202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.688202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.688254 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.688542 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.688544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.688544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.688668 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:27.688737 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:27.689084 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 08:42:27.689159 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 08:42:27.689561 8284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:27.679666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:32787 (local address 127.31.63.66:47429)
0809 08:42:27.679864 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:27.679920 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:27.680899 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:27.681137 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:27.681140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:27.681408 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:27.681690 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:27.681695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.682362 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.682363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:27.683167 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:27.683169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.683507 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.683508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:27.683612 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:27.684193 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:27.684202 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:27.685995 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:27.687779 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:27.687788 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:27.687790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:27.688121 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:27.688322 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:27.688323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:27.688324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:27.688475 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:27.688759 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:27.688767 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:27.688938 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 08:42:27.689059 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 08:42:27.689249 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":101,"threads_started":1}
I20260809 08:42:27.690200 8282 heartbeater.cc:347] Connected to a master server at 127.31.63.126:32787
I20260809 08:42:27.690302 8282 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:27.690505 8282 heartbeater.cc:511] Master 127.31.63.126:32787 requested a full tablet report, sending...
I20260809 08:42:27.690943 7816 ts_manager.cc:194] Registered new tserver with Master: aa295c6ab4a94b9abf33b060c7d523e7 (127.31.63.66:46301)
I20260809 08:42:27.691340 7816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:47429
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:42:27.713619 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:32787
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:45543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:27.809715 8287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:27.809929 8287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:27.809953 8287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:27.811170 8287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:27.811221 8287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:27.811250 8287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:27.812806 8287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:45543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:32787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:27.813071 8287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:27.813336 8287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:27.818610 8295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:27.818601 8299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:27.818624 8294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:27.818823 8287 server_base.cc:1034] running on GCE node
I20260809 08:42:27.819157 8287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:27.819427 8287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:27.820590 8287 hybrid_clock.cc:648] HybridClock initialized: now 1786264947820578 us; error 31 us; skew 500 ppm
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.823310 8287 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:42:27.823724 8287 webserver.cc:460] Webserver started at http://127.31.63.67:39553/ using document root <none> and password file <none>
I20260809 08:42:27.823908 8287 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:27.823949 8287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:27.824043 8287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:27.825068 8287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "8e5a3affdd854d41899e80a49e4ee8f1"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "4ebd3cf92f1bbf5f97c78fec0807fdca"
server_key_iv: "68922528d4dee7f1c495d9c3463c07c7"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.825410 8287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "8e5a3affdd854d41899e80a49e4ee8f1"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "4ebd3cf92f1bbf5f97c78fec0807fdca"
server_key_iv: "68922528d4dee7f1c495d9c3463c07c7"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.827152 8287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:27.827965 8309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:27.828150 8287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:27.828244 8287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "8e5a3affdd854d41899e80a49e4ee8f1"
format_stamp: "Formatted at 2026-08-09 08:42:27 on dist-test-slave-gkw7"
server_key: "4ebd3cf92f1bbf5f97c78fec0807fdca"
server_key_iv: "68922528d4dee7f1c495d9c3463c07c7"
server_key_version: "encryptionkey@0"
I20260809 08:42:27.828302 8287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:27.849879 8287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:27.852213 8287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:27.852332 8287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:27.852553 8287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:27.852905 8287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:27.852941 8287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:27.852964 8287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:27.853012 8287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:27.859386 8287 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:46241
I20260809 08:42:27.859433 8464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:46241 every 8 connection(s)
I20260809 08:42:27.859783 8287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:42:27.860188 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 8287
I20260809 08:42:27.860282 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:27.860440 31997 external_mini_cluster.cc:1428] Setting key 649716d305319575bdeda5c6222dd7e0
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:42:27.869241 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:27.860566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:59039 (local address 127.31.63.126:32787)
0809 08:42:27.860635 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:27.860637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:27.860874 (+ 237us) server_negotiation.cc:407] Connection header received
0809 08:42:27.862076 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:27.862077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:27.862088 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:27.862137 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:27.862738 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.863240 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.864321 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.864508 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.864562 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:27.866886 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:27.866888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:27.866888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:27.866895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:27.868051 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.868427 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.868428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.868428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.868487 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.868713 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.868714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.868714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.868826 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:27.868877 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:27.869089 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 08:42:27.869155 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:42:27.869462 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:27.860524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:32787 (local address 127.31.63.67:59039)
0809 08:42:27.860745 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:27.860797 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:27.861984 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:27.862200 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:27.862203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:27.862463 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:27.862665 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:27.862669 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.863322 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.863323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:27.864252 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:27.864254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.864582 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.864583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:27.864681 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:27.865279 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:27.865288 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:27.866803 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:27.868121 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:27.868127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:27.868129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:27.868355 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:27.868558 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:27.868558 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:27.868559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:27.868657 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:27.868867 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:27.868869 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:27.868980 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 08:42:27.869058 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:42:27.869179 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":109,"threads_started":1}
I20260809 08:42:27.870074 8466 heartbeater.cc:347] Connected to a master server at 127.31.63.126:32787
I20260809 08:42:27.870195 8466 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:27.870389 8466 heartbeater.cc:511] Master 127.31.63.126:32787 requested a full tablet report, sending...
I20260809 08:42:27.870808 7816 ts_manager.cc:194] Registered new tserver with Master: 8e5a3affdd854d41899e80a49e4ee8f1 (127.31.63.67:46241)
I20260809 08:42:27.871196 7816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:59039
I20260809 08:42:27.872298 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/test-admin.keytab.
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.889797 31997 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 08:42:27.891459 8478 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:42:27.890708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.31.63.65:33467)
0809 08:42:27.890900 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:27.890905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:27.890916 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:42:27.890944 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:27.890946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:27.890960 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:27.891028 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:27.891361 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35722: BlockingRecv error: recv got EOF from 127.0.0.1:35722 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":106,"threads_started":1}
W20260809 08:42:27.891523 31997 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5cc' of type FILE
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.892496 31997 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.893402 31997 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:42:27.901139 8478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:27.893997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35732 (local address 127.31.63.65:33467)
0809 08:42:27.894053 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:27.894055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:27.894084 (+ 29us) server_negotiation.cc:407] Connection header received
0809 08:42:27.894137 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:27.894138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:27.894148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:27.894194 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:27.894570 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.895336 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.896101 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.896473 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.896530 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:27.898554 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:27.898562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:27.898562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:27.898569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:27.900143 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.900468 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.900471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.900471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.900526 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.900689 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.900690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.900690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.900830 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:27.900950 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:27.901005 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 08:42:27.901065 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 08:42:27.903196 8478 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:42:27.902653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.31.63.65:33467)
0809 08:42:27.902698 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:27.902699 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:27.902776 (+ 77us) server_negotiation.cc:407] Connection header received
0809 08:42:27.902812 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:27.902813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:27.902821 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:27.902869 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:27.903141 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35748: BlockingRecv error: recv got EOF from 127.0.0.1:35748 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:42:27.903309 31997 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5cc' of type FILE
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.904274 31997 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.905120 31997 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:42:27.912299 8478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:27.905680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.31.63.65:33467)
0809 08:42:27.905743 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:27.905745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:27.905816 (+ 71us) server_negotiation.cc:407] Connection header received
0809 08:42:27.905889 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:27.905890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:27.905901 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:27.905970 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:27.906357 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.906840 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.907551 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.907760 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.907838 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:27.909893 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:27.909894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:27.909895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:27.909902 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:27.911161 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.911527 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.911528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.911528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.911589 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.911800 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.911801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.911802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.911902 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:27.911997 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:27.912108 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 08:42:27.912190 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 08:42:27.913904 8478 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:42:27.913323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35760 (local address 127.31.63.65:33467)
0809 08:42:27.913375 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:27.913378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:27.913427 (+ 49us) server_negotiation.cc:407] Connection header received
0809 08:42:27.913499 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:27.913500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:27.913509 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:27.913591 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:27.913856 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35760: BlockingRecv error: recv got EOF from 127.0.0.1:35760 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 08:42:27.914063 31997 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786264918773806-31997-0/krb5kdc/krb5cc)
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.915002 31997 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:27.915840 31997 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 08:42:27 dist-test-slave-gkw7 krb5kdc[7755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:42:27.922645 8478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:27.916333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.31.63.65:33467)
0809 08:42:27.916387 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:27.916389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:27.916396 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:42:27.916456 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:27.916457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:27.916466 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:27.916515 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:27.916896 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.917365 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.918129 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:27.918341 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:27.918427 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:27.920322 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:27.920323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:27.920324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:27.920331 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:27.921449 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.921884 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.921886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.921886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.921946 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:27.922146 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:27.922147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:27.922148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:27.922256 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:27.922355 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:27.922473 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 08:42:27.922550 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:42:27.923290 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 7912
I20260809 08:42:27.938253 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 8100
I20260809 08:42:27.945227 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 8287
I20260809 08:42:27.953967 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 7773
2026-08-09T08:42:27Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2118 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:42:28 dist-test-slave-gkw7 krb5kdc[8496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:42:28 dist-test-slave-gkw7 krb5kdc[8496](info): set up 2 sockets
Aug 09 08:42:28 dist-test-slave-gkw7 krb5kdc[8496](info): commencing operation
krb5kdc: starting...
W20260809 08:42:29.215492 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:42:29 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:42:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.126.keytab.
Entry for principal oryx/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.126.keytab.
I20260809 08:42:29.344465 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33565
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.126.keytab
--principal=oryx/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:29.432544 8514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:29.432777 8514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:29.432801 8514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:29.434082 8514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:29.434128 8514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:29.434144 8514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:29.434156 8514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:29.434192 8514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:29.435737 8514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.126.keytab
--principal=oryx/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:29.436020 8514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:29.436298 8514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:29.441485 8521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:29.441628 8528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:29.441740 8514 server_base.cc:1034] running on GCE node
W20260809 08:42:29.441481 8520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:29.442303 8514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:29.442631 8514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:29.443776 8514 hybrid_clock.cc:648] HybridClock initialized: now 1786264949443763 us; error 31 us; skew 500 ppm
Aug 09 08:42:29 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:29.446487 8514 init.cc:377] Logged in from keytab as oryx/127.31.63.126@KRBTEST.COM (short username oryx)
I20260809 08:42:29.447000 8514 webserver.cc:460] Webserver started at http://127.31.63.126:39175/ using document root <none> and password file <none>
I20260809 08:42:29.447247 8514 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:29.447319 8514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:29.447479 8514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:29.448586 8514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "021edbb7538f4488976e2bbed5f7f747"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "d85ddeec91b5b47cb1866eccf192562d"
server_key_iv: "a3f4853087ca4252384d26dd486f9fab"
server_key_version: "encryptionkey@0"
I20260809 08:42:29.448985 8514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "021edbb7538f4488976e2bbed5f7f747"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "d85ddeec91b5b47cb1866eccf192562d"
server_key_iv: "a3f4853087ca4252384d26dd486f9fab"
server_key_version: "encryptionkey@0"
I20260809 08:42:29.450732 8514 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 08:42:29.451658 8539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:29.451839 8514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:29.451936 8514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "021edbb7538f4488976e2bbed5f7f747"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "d85ddeec91b5b47cb1866eccf192562d"
server_key_iv: "a3f4853087ca4252384d26dd486f9fab"
server_key_version: "encryptionkey@0"
I20260809 08:42:29.452014 8514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:29.467787 8514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:29.470300 8514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:29.470423 8514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:29.474710 8514 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:33565
I20260809 08:42:29.474768 8613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:33565 every 8 connection(s)
I20260809 08:42:29.475147 8514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:42:29.476238 8616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:29.478713 8616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747: Bootstrap starting.
I20260809 08:42:29.479342 8616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:29.479646 8616 log.cc:824] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:29.480484 8616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747: No bootstrap required, opened a new log
I20260809 08:42:29.480484 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 8514
I20260809 08:42:29.480597 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:42:29.480794 31997 external_mini_cluster.cc:1428] Setting key f277f4c6bb9f9e569bac44e6dbb87c07
I20260809 08:42:29.482525 8616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } }
I20260809 08:42:29.482699 8616 raft_consensus.cc:374] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:29.482748 8616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021edbb7538f4488976e2bbed5f7f747, State: Initialized, Role: FOLLOWER
I20260809 08:42:29.482884 8616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } }
I20260809 08:42:29.482983 8616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:42:29.483026 8616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:42:29.483080 8616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:29.484000 8616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } }
I20260809 08:42:29.484125 8616 leader_election.cc:302] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 021edbb7538f4488976e2bbed5f7f747; no voters:
I20260809 08:42:29.484328 8616 leader_election.cc:288] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:42:29.484370 8626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:29.484499 8626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 1 LEADER]: Becoming Leader. State: Replica: 021edbb7538f4488976e2bbed5f7f747, State: Running, Role: LEADER
I20260809 08:42:29.484604 8616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:29.484582 8626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } }
I20260809 08:42:29.484939 8630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "021edbb7538f4488976e2bbed5f7f747" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } }
I20260809 08:42:29.485026 8630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:29.485692 8635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:42:29.485294 8631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 021edbb7538f4488976e2bbed5f7f747. Latest consensus state: current_term: 1 leader_uuid: "021edbb7538f4488976e2bbed5f7f747" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } }
I20260809 08:42:29.486043 8631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:29.486368 8635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:42:29.488183 8635 catalog_manager.cc:1347] Generated new cluster ID: aa58af3af45f4db4addcaa8215477801
I20260809 08:42:29.488237 8635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:42:29 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:42:29.493492 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:29.481354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.31.63.126:33565)
0809 08:42:29.481600 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:29.481610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:29.481645 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:42:29.482507 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:29.482514 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:29.482581 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:29.482729 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:29.483510 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:29.484685 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:29.485858 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:29.486480 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:29.486943 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:29.489452 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:29.489456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:29.489457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:29.489476 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:29.491434 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:29.491872 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:29.491876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:29.491876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:29.492024 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:29.492482 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:29.492484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:29.492484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:29.492779 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:29.492975 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:29.493086 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 08:42:29.493251 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":101,"threads_started":1}
I20260809 08:42:29.526556 8635 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:42:29.527364 8635 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:42:29.535593 8635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747: Generated new TSK 0
I20260809 08:42:29.535912 8635 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab.
Entry for principal oryx/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab.
I20260809 08:42:29.577080 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33565
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab
--principal=oryx/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:29.669661 8655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:29.669888 8655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:29.669914 8655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:29.671101 8655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:29.671144 8655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:29.671171 8655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:29.672703 8655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab
--principal=oryx/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:29.672967 8655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:29.673172 8655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:29.678429 8664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:29.678452 8662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:29.678431 8660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:29.678730 8655 server_base.cc:1034] running on GCE node
I20260809 08:42:29.678912 8655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:29.679122 8655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:29.680279 8655 hybrid_clock.cc:648] HybridClock initialized: now 1786264949680255 us; error 48 us; skew 500 ppm
Aug 09 08:42:29 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:29.683121 8655 init.cc:377] Logged in from keytab as oryx/127.31.63.65@KRBTEST.COM (short username oryx)
I20260809 08:42:29.683595 8655 webserver.cc:460] Webserver started at http://127.31.63.65:41125/ using document root <none> and password file <none>
I20260809 08:42:29.683845 8655 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:29.683893 8655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:29.684089 8655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:29.685263 8655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "f4009453748648e8aa37ea67947bcb00"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "7aa1823581771d12c4f89e5382802fe9"
server_key_iv: "34dd34355083a38d52069c832573781c"
server_key_version: "encryptionkey@0"
I20260809 08:42:29.685683 8655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "f4009453748648e8aa37ea67947bcb00"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "7aa1823581771d12c4f89e5382802fe9"
server_key_iv: "34dd34355083a38d52069c832573781c"
server_key_version: "encryptionkey@0"
I20260809 08:42:29.687206 8655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:29.688122 8672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:29.688338 8655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:29.688452 8655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "f4009453748648e8aa37ea67947bcb00"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "7aa1823581771d12c4f89e5382802fe9"
server_key_iv: "34dd34355083a38d52069c832573781c"
server_key_version: "encryptionkey@0"
I20260809 08:42:29.688531 8655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:29.700958 8655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:29.703509 8655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:29.703687 8655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:29.703949 8655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:29.704303 8655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:29.704370 8655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:29.704404 8655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:29.704437 8655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:29.711062 8655 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:34803
I20260809 08:42:29.711117 8836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:34803 every 8 connection(s)
I20260809 08:42:29.711460 8655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:29.713555 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 8655
I20260809 08:42:29.713675 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:42:29.713898 31997 external_mini_cluster.cc:1428] Setting key 508ba81fab5d3738eed2b479a8aa05c3
Aug 09 08:42:29 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:42:29.722848 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:29.712238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:60005 (local address 127.31.63.126:33565)
0809 08:42:29.712285 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:29.712287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:29.712494 (+ 207us) server_negotiation.cc:407] Connection header received
0809 08:42:29.712999 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:29.713000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:29.713015 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:29.713070 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:29.714270 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:29.714955 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:29.716545 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:29.716907 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:29.716984 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:29.719389 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:29.719392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:29.719392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:29.719405 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:29.721207 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:29.721742 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:29.721744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:29.721744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:29.721834 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:29.722112 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:29.722113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:29.722113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:29.722273 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:29.722351 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:29.722628 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 08:42:29.722739 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:42:29.722889 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:29.712217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:60005)
0809 08:42:29.712385 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:29.712422 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:29.712912 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:29.713130 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:29.713133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:29.713877 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:29.714147 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:29.714152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:29.715066 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:29.715068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:29.716437 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:29.716441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:29.717007 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:29.717008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:29.717132 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:29.717740 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:29.717749 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:29.719252 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:29.721323 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:29.721330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:29.721332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:29.721660 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:29.721923 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:29.721923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:29.721924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:29.722032 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:29.722346 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:29.722351 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:29.722476 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 08:42:29.722558 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 08:42:29.722713 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260809 08:42:29.723762 8837 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33565
I20260809 08:42:29.723884 8837 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.31.63.66@KRBTEST.COM; defaulting to no policy
I20260809 08:42:29.724097 8837 heartbeater.cc:511] Master 127.31.63.126:33565 requested a full tablet report, sending...
I20260809 08:42:29.724700 8568 ts_manager.cc:194] Registered new tserver with Master: f4009453748648e8aa37ea67947bcb00 (127.31.63.65:34803)
I20260809 08:42:29.725343 8568 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:60005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab.
Entry for principal oryx/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab.
I20260809 08:42:29.745740 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33565
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab
--principal=oryx/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:29.831622 8843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:29.831831 8843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:29.831856 8843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:29.832958 8843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:29.832993 8843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:29.833017 8843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:29.834537 8843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab
--principal=oryx/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:29.834733 8843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:29.834914 8843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:29.840265 8853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:29.840287 8851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:29.840322 8850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:29.840472 8843 server_base.cc:1034] running on GCE node
I20260809 08:42:29.840680 8843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:29.840983 8843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:29.842180 8843 hybrid_clock.cc:648] HybridClock initialized: now 1786264949842158 us; error 38 us; skew 500 ppm
Aug 09 08:42:29 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:29.844669 8843 init.cc:377] Logged in from keytab as oryx/127.31.63.66@KRBTEST.COM (short username oryx)
I20260809 08:42:29.845119 8843 webserver.cc:460] Webserver started at http://127.31.63.66:46411/ using document root <none> and password file <none>
I20260809 08:42:29.845351 8843 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:29.845424 8843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:29.845602 8843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:29.846673 8843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "a3e27e0401e54d88a1217cbe1ff301e8"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "0c3f6961c1474fd71aadb2755c0141c6"
server_key_iv: "f48dd97927302169a661bfbc35dba38a"
server_key_version: "encryptionkey@0"
I20260809 08:42:29.847052 8843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "a3e27e0401e54d88a1217cbe1ff301e8"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "0c3f6961c1474fd71aadb2755c0141c6"
server_key_iv: "f48dd97927302169a661bfbc35dba38a"
server_key_version: "encryptionkey@0"
I20260809 08:42:29.848433 8843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:42:29.849205 8865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:29.849411 8843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:29.849510 8843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "a3e27e0401e54d88a1217cbe1ff301e8"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "0c3f6961c1474fd71aadb2755c0141c6"
server_key_iv: "f48dd97927302169a661bfbc35dba38a"
server_key_version: "encryptionkey@0"
I20260809 08:42:29.849637 8843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:29.859987 8843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:29.862080 8843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:29.862237 8843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:29.862495 8843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:29.862831 8843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:29.862886 8843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:29.862927 8843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:29.862951 8843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:29.869251 8843 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:41417
I20260809 08:42:29.869387 9024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:41417 every 8 connection(s)
I20260809 08:42:29.869692 8843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:42:29.871716 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 8843
I20260809 08:42:29.871798 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:42:29.872018 31997 external_mini_cluster.cc:1428] Setting key 2615434beb6d65fd3087985f762b6bec
Aug 09 08:42:29 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:42:29.879917 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:29.870475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:60061 (local address 127.31.63.126:33565)
0809 08:42:29.870536 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:29.870538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:29.870763 (+ 225us) server_negotiation.cc:407] Connection header received
0809 08:42:29.871309 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:29.871311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:29.871325 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:29.871376 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:29.872473 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:29.873005 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:29.874106 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:29.874336 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:29.874422 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:29.876936 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:29.876937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:29.876938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:29.876946 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:29.878486 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:29.878900 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:29.878902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:29.878902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:29.878970 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:29.879229 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:29.879230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:29.879230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:29.879364 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:29.879442 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:29.879729 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 08:42:29.879821 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:42:29.880116 9031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:29.870475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:60061)
0809 08:42:29.870638 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:29.870676 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:29.871195 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:29.871445 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:29.871449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:29.872164 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:29.872373 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:29.872378 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:29.873118 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:29.873119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:29.874015 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:29.874017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:29.874432 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:29.874433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:29.874600 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:29.875212 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:29.875223 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:29.876833 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:29.878578 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:29.878586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:29.878588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:29.878822 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:29.879039 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:29.879040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:29.879041 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:29.879151 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:29.879441 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:29.879450 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:29.879590 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 08:42:29.879692 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 08:42:29.879868 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260809 08:42:29.880577 9026 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33565
I20260809 08:42:29.880687 9026 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:29.880900 9026 heartbeater.cc:511] Master 127.31.63.126:33565 requested a full tablet report, sending...
I20260809 08:42:29.881300 8568 ts_manager.cc:194] Registered new tserver with Master: a3e27e0401e54d88a1217cbe1ff301e8 (127.31.63.66:41417)
I20260809 08:42:29.881752 8568 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:60061
WARNING: no policy specified for oryx/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab.
Entry for principal oryx/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab.
I20260809 08:42:29.901515 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33565
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab
--principal=oryx/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:29.987257 9034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:29.987459 9034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:29.987483 9034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:29.988555 9034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:29.988590 9034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:29.988615 9034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:29.990065 9034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab
--principal=oryx/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:29.990260 9034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:29.990439 9034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:29.995994 9044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:29.996055 9046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:29.996011 9034 server_base.cc:1034] running on GCE node
W20260809 08:42:29.995994 9043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:29.996519 9034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:29.996820 9034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:29.998021 9034 hybrid_clock.cc:648] HybridClock initialized: now 1786264949998005 us; error 26 us; skew 500 ppm
Aug 09 08:42:29 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:30.000808 9034 init.cc:377] Logged in from keytab as oryx/127.31.63.67@KRBTEST.COM (short username oryx)
I20260809 08:42:30.001307 9034 webserver.cc:460] Webserver started at http://127.31.63.67:34425/ using document root <none> and password file <none>
I20260809 08:42:30.001502 9034 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:30.001605 9034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:30.001716 9034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:30.002781 9034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "1373576645147ec92b97395e3f52b5f2"
server_key_iv: "e5b888f5937e079e1356e41720203963"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.003098 9034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "1373576645147ec92b97395e3f52b5f2"
server_key_iv: "e5b888f5937e079e1356e41720203963"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.004460 9034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:42:30.005352 9056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.005574 9034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:30.005669 9034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "1373576645147ec92b97395e3f52b5f2"
server_key_iv: "e5b888f5937e079e1356e41720203963"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.005734 9034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:30.018651 9034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:30.020612 9034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:30.020740 9034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:30.020941 9034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:30.021210 9034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:30.021245 9034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.021267 9034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:30.021281 9034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.027789 9034 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:36907
I20260809 08:42:30.027853 9212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:36907 every 8 connection(s)
I20260809 08:42:30.028174 9034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:42:30.037509 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 9034
I20260809 08:42:30.037652 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:42:30.037859 31997 external_mini_cluster.cc:1428] Setting key 39597d4c6f3e54e301bd137415789fd8
I20260809 08:42:30.038925 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.029000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:38637 (local address 127.31.63.126:33565)
0809 08:42:30.029072 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.029074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.029345 (+ 271us) server_negotiation.cc:407] Connection header received
0809 08:42:30.029863 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.029864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.029876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.029937 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.030884 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.031393 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.032538 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.032736 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.032794 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.035407 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.035409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.035410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.035419 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.037036 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.037562 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.037564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.037564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.037656 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.038025 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.038027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.038027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.038238 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.038321 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.038729 (+ 408us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.038817 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:42:30.039171 9216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.028966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:38637)
0809 08:42:30.029204 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.029248 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.029776 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.029990 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.029993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.030598 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.030801 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.030806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.031506 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.031507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.032442 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.032445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.032826 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.032827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.032958 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.033647 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.033656 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.035271 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.037166 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.037174 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.037176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.037448 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.037749 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.037751 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.037751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.037907 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.038382 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.038391 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.038530 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.038679 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.038865 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":162,"threads_started":1}
I20260809 08:42:30.039733 9213 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33565
I20260809 08:42:30.039856 9213 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:30.040062 9213 heartbeater.cc:511] Master 127.31.63.126:33565 requested a full tablet report, sending...
I20260809 08:42:30.040491 8568 ts_manager.cc:194] Registered new tserver with Master: 0d14dbfb74a142e5a9e5b9552ccabfa0 (127.31.63.67:36907)
I20260809 08:42:30.040869 8568 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:38637
I20260809 08:42:30.049957 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.63.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.125.keytab.
Entry for principal oryx/127.31.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.125.keytab.
I20260809 08:42:30.078344 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.125:43031
--webserver_interface=127.31.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.125.keytab
--principal=oryx/127.31.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33565,127.31.63.125:43031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:30.162556 9220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:30.162811 9220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:30.162851 9220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:30.164014 9220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:30.164088 9220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:30.164115 9220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:30.164147 9220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:30.164171 9220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:30.165808 9220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33565,127.31.63.125:43031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.125.keytab
--principal=oryx/127.31.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.125:43031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
W20260809 08:42:30.165977 9220 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.63.126:33565,127.31.63.125:43031'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 08:42:30.166123 9220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:30.166349 9220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:30.171645 9229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:30.171674 9226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:30.171617 9227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:30.171839 9220 server_base.cc:1034] running on GCE node
I20260809 08:42:30.172106 9220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:30.172405 9220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:30.173606 9220 hybrid_clock.cc:648] HybridClock initialized: now 1786264950173580 us; error 40 us; skew 500 ppm
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:30.176101 9220 init.cc:377] Logged in from keytab as oryx/127.31.63.125@KRBTEST.COM (short username oryx)
I20260809 08:42:30.176594 9220 webserver.cc:460] Webserver started at http://127.31.63.125:39039/ using document root <none> and password file <none>
I20260809 08:42:30.176864 9220 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:30.176936 9220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:30.177099 9220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:30.178305 9220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data/instance:
uuid: "d191b1bebc934a6ba4251a1055cf4bb3"
format_stamp: "Formatted at 2026-08-09 08:42:30 on dist-test-slave-gkw7"
server_key: "1ea9b1bfccb76a88893ae7aa514b4d4b"
server_key_iv: "2df51e3116b7d56fbe93b94f52be693c"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.178699 9220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal/instance:
uuid: "d191b1bebc934a6ba4251a1055cf4bb3"
format_stamp: "Formatted at 2026-08-09 08:42:30 on dist-test-slave-gkw7"
server_key: "1ea9b1bfccb76a88893ae7aa514b4d4b"
server_key_iv: "2df51e3116b7d56fbe93b94f52be693c"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.180145 9220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:42:30.181007 9240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.181222 9220 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:30.181334 9220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal
uuid: "d191b1bebc934a6ba4251a1055cf4bb3"
format_stamp: "Formatted at 2026-08-09 08:42:30 on dist-test-slave-gkw7"
server_key: "1ea9b1bfccb76a88893ae7aa514b4d4b"
server_key_iv: "2df51e3116b7d56fbe93b94f52be693c"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.181408 9220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:30.210287 9220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:30.212268 9220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:30.212436 9220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:30.218727 9220 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.125:43031
I20260809 08:42:30.218778 9312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.125:43031 every 8 connection(s)
I20260809 08:42:30.219175 9220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
I20260809 08:42:30.220359 9313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:30.222049 9313 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:42:30.224089 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 9220
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.125@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.63.125@KRBTEST.COM
I20260809 08:42:30.232025 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.222754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54384 (local address 127.31.63.126:33565)
0809 08:42:30.222821 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.222823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.223025 (+ 202us) server_negotiation.cc:407] Connection header received
0809 08:42:30.223586 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.223587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.223598 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.223657 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.224348 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.225013 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.226312 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.226503 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.226556 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.228986 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.228989 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.228989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.229001 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.230497 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.230958 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.230959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.230960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.231027 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.231325 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.231326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.231326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.231470 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.231553 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.231832 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.231913 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:42:30.232278 9317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.222707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:54384)
0809 08:42:30.222891 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.222934 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.223497 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.223718 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.223721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.223966 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.224225 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.224231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.225129 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.225130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.226228 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.226231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.226578 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.226579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.226711 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.227274 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.227283 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.228895 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.230577 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.230585 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.230587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.230874 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.231107 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.231108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.231109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.231251 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.231560 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.231567 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.231689 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.231808 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.232020 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260809 08:42:30.233054 9313 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:42:30.234719 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.224774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.31.63.125:43031)
0809 08:42:30.224926 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.224931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.224973 (+ 42us) server_negotiation.cc:407] Connection header received
0809 08:42:30.225036 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.225038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.225078 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.225165 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.225800 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.226747 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.227449 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.227907 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.227981 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.231419 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.231426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.231427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.231433 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.233452 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.233892 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.233894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.233895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.233946 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.234166 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.234167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.234167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.234328 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.234479 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.234528 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.234602 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.125@KRBTEST.COM for oryx/127.31.63.125@KRBTEST.COM
I20260809 08:42:30.240517 9317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.233346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:43031 (local address 127.0.0.1:51252)
0809 08:42:30.233392 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.233404 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.233443 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.233708 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.233709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.233864 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.233940 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.233942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.234933 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.234934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.235552 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.235554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.235894 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.235894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.235947 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.236338 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.236344 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.237461 (+ 1117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.239563 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.239563 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.239564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.239732 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.239984 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.239985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.239985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.240038 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.240289 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.240290 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.240325 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.240375 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.240452 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:42:30.240661 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.233408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51252 (local address 127.31.63.125:43031)
0809 08:42:30.233601 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.233602 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.233607 (+ 5us) server_negotiation.cc:407] Connection header received
0809 08:42:30.233621 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.233621 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.233628 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.233678 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.234039 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.234842 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.235631 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.235828 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.235884 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.237578 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.237579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.237580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.237597 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.239486 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.239855 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.239856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.239857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.239916 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.240117 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.240118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.240118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.240231 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.240289 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.240409 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.240505 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":175,"thread_start_us":97,"threads_started":1}
I20260809 08:42:30.242120 9313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: Bootstrap starting.
I20260809 08:42:30.242803 9313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:30.243111 9313 log.cc:824] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:30.244026 9313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: No bootstrap required, opened a new log
I20260809 08:42:30.245486 9313 raft_consensus.cc:348] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } }
I20260809 08:42:30.245707 9313 raft_consensus.cc:374] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:30.245788 9313 raft_consensus.cc:729] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d191b1bebc934a6ba4251a1055cf4bb3, State: Initialized, Role: FOLLOWER
I20260809 08:42:30.245934 9313 consensus_queue.cc:260] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } }
I20260809 08:42:30.246290 9313 sys_catalog.cc:564] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:30.246263 9323 sys_catalog.cc:455] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } } }
I20260809 08:42:30.246384 9323 sys_catalog.cc:458] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:42:30.248709 9341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:42:30.248821 9341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:42:30.250811 9220 master_runner.cc:428] Detected that this master d191b1bebc934a6ba4251a1055cf4bb3 is joining an existing cluster
I20260809 08:42:30.250905 9220 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.63.125:43031
I20260809 08:42:30.259238 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.252131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.31.63.126:33565)
0809 08:42:30.252180 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.252183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.252440 (+ 257us) server_negotiation.cc:407] Connection header received
0809 08:42:30.252491 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.252492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.252504 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.252555 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.252990 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.253495 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.254347 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.254555 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.254638 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.255743 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.255745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.255745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.255752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.257741 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.258229 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.258231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.258231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.258298 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.258572 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.258573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.258574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.258716 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.258793 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.259037 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.259141 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:42:30.259275 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.252036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:54396)
0809 08:42:30.252377 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.252384 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.252441 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.252556 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.252557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.252786 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.252841 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.252843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.253666 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.253667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.254243 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.254245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.254675 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.254676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.254734 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.255298 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.255302 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.255622 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.257814 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.257816 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.257816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.258139 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.258383 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.258384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.258385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.258462 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.258800 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.258801 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.258852 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.258990 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.259157 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260809 08:42:30.259769 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.252035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:43031 (local address 127.0.0.1:51264)
0809 08:42:30.252235 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.252247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.252309 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.252487 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.252488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.252759 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.252848 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.252850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.253799 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.253800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.254598 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.254601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.255672 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.255673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.255734 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.256183 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.256188 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.256643 (+ 455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.258503 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.258505 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.258505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.258739 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.258966 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.258967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.258967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.259025 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.259323 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.259324 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.259459 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.259532 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.259630 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":110,"threads_started":1}
I20260809 08:42:30.259838 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.252214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.31.63.125:43031)
0809 08:42:30.252257 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.252259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.252297 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:42:30.252354 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.252355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.252367 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.252440 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.252947 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.253697 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.255241 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.255546 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.255595 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.256735 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.256736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.256736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.256743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.258421 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.258839 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.258840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.258841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.258891 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.259098 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.259098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.259099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.259237 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.259281 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.259611 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.259697 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.126@KRBTEST.COM for oryx/127.31.63.125@KRBTEST.COM
I20260809 08:42:30.269892 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.262396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.31.63.125:43031)
0809 08:42:30.262435 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.262437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.262637 (+ 200us) server_negotiation.cc:407] Connection header received
0809 08:42:30.262737 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.262738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.262747 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.262793 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.263310 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.263752 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.264863 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.265037 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.265115 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.267289 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.267290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.267290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.267297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.268669 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.269055 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.269056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.269056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.269115 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.269315 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.269316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.269316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.269421 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.269496 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.269719 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.269798 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:42:30.269901 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.262310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:43031 (local address 127.0.0.1:51280)
0809 08:42:30.262507 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.262559 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.262659 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.262799 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.262802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.263141 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.263202 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.263205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.263836 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.263837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.264778 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.264781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.265139 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.265140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.265222 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.265728 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.265733 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.267196 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.268767 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.268775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.268777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.268977 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.269182 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.269183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.269183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.269241 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.269489 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.269496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.269580 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.269679 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.269801 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260809 08:42:30.270576 8568 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.63.125:43031: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3"
member_type: NON_VOTER
last_known_addr {
host: "127.31.63.125"
port: 43031
}
attrs {
promote: true
}
}
dest_uuid: "021edbb7538f4488976e2bbed5f7f747"
cas_config_opid_index: -1
I20260809 08:42:30.270946 8568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } }
I20260809 08:42:30.271482 9355 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d191b1bebc934a6ba4251a1055cf4bb3 (127.31.63.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } } }
I20260809 08:42:30.271852 9279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:30.272615 9355 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.63.125:43031
I20260809 08:42:30.272871 9279 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Refusing update from remote peer 021edbb7538f4488976e2bbed5f7f747: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 08:42:30.272856 9352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "021edbb7538f4488976e2bbed5f7f747" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } } }
I20260809 08:42:30.272928 9352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:30.273118 9220 master.cc:537] Master@127.31.63.125:43031 shutting down...
I20260809 08:42:30.273401 9352 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 08:42:30.274394 8546 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 08:42:30.275394 8546 consensus_peers.cc:487] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 -> Peer d191b1bebc934a6ba4251a1055cf4bb3 (127.31.63.125:43031): Couldn't send request to peer d191b1bebc934a6ba4251a1055cf4bb3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 08:42:30.276111 9220 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:30.276201 9220 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:30.276265 9220 tablet_replica.cc:330] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: stopping tablet replica
I20260809 08:42:30.289337 9220 master.cc:552] Master@127.31.63.125:43031 shutdown complete.
I20260809 08:42:30.289386 9220 master_runner.cc:305] Clearing existing system tablet
I20260809 08:42:30.289543 9220 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:30.289637 9220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:30.290293 9220 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:42:30.290802 9362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.290916 9220 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:30.290974 9220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:42:30.290997 9220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal
uuid: "d191b1bebc934a6ba4251a1055cf4bb3"
format_stamp: "Formatted at 2026-08-09 08:42:30 on dist-test-slave-gkw7"
server_key: "1ea9b1bfccb76a88893ae7aa514b4d4b"
server_key_iv: "2df51e3116b7d56fbe93b94f52be693c"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.292270 9220 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:30.293730 9220 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 08:42:30.293792 9220 log.cc:1196] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 08:42:30.293998 9220 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: Deleting consensus metadata
I20260809 08:42:30.294126 9220 master_runner.cc:315] Copying system tablet from 127.31.63.126:33565
I20260809 08:42:30.294793 9220 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: tablet copy: Beginning tablet copy session from remote peer at address 127.31.63.126:33565
I20260809 08:42:30.301416 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.295301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54402 (local address 127.31.63.126:33565)
0809 08:42:30.295357 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.295359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.295429 (+ 70us) server_negotiation.cc:407] Connection header received
0809 08:42:30.295502 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.295502 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.295512 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.295561 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.295943 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.296434 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.297495 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.297769 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.297824 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.298901 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.298903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.298903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.298909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.300331 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.300719 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.300720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.300720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.300790 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.300986 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.300986 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.300987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.301085 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.301136 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.301294 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.301352 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:42:30.301493 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.295211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:54402)
0809 08:42:30.295358 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.295371 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.295440 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.295591 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.295592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.295796 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.295869 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.295871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.296512 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.296513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.297406 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.297408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.297845 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.297846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.297916 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.298405 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.298411 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.298816 (+ 405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.300414 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.300415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.300416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.300645 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.300857 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.300857 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.300858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.300925 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.301151 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.301152 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.301200 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.301279 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.301381 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260809 08:42:30.302162 8593 tablet_copy_service.cc:140] P 021edbb7538f4488976e2bbed5f7f747: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d191b1bebc934a6ba4251a1055cf4bb3 ({username='oryx', principal='oryx/127.31.63.125@KRBTEST.COM'} at 127.0.0.1:54402)
I20260809 08:42:30.302311 8593 tablet_copy_service.cc:161] P 021edbb7538f4488976e2bbed5f7f747: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d191b1bebc934a6ba4251a1055cf4bb3 at {username='oryx', principal='oryx/127.31.63.125@KRBTEST.COM'} at 127.0.0.1:54402: session id = d191b1bebc934a6ba4251a1055cf4bb3-00000000000000000000000000000000
I20260809 08:42:30.303179 8593 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 08:42:30.303819 9220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 08:42:30.304843 9220 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: tablet copy: Starting download of 0 data blocks...
I20260809 08:42:30.305008 9220 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: tablet copy: Starting download of 1 WAL segments...
I20260809 08:42:30.306113 9220 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 08:42:30.306907 8593 tablet_copy_service.cc:342] P 021edbb7538f4488976e2bbed5f7f747: Request end of tablet copy session d191b1bebc934a6ba4251a1055cf4bb3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.63.125@KRBTEST.COM'} at 127.0.0.1:54402
I20260809 08:42:30.306969 8593 tablet_copy_service.cc:433] P 021edbb7538f4488976e2bbed5f7f747: ending tablet copy session d191b1bebc934a6ba4251a1055cf4bb3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d191b1bebc934a6ba4251a1055cf4bb3
I20260809 08:42:30.309878 9220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:30.311062 9371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:30.311167 9381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:30.311208 9373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:30.311359 9220 server_base.cc:1034] running on GCE node
I20260809 08:42:30.311528 9220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:30.311700 9220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:30.312855 9220 hybrid_clock.cc:648] HybridClock initialized: now 1786264950312844 us; error 25 us; skew 500 ppm
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:30.314437 9220 init.cc:377] Logged in from keytab as oryx/127.31.63.125@KRBTEST.COM (short username oryx)
I20260809 08:42:30.314771 9220 webserver.cc:460] Webserver started at http://127.31.63.125:34873/ using document root <none> and password file <none>
I20260809 08:42:30.314890 9220 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:30.314945 9220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:30.315536 9220 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:42:30.316111 9391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.316259 9220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:42:30.316318 9220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal
uuid: "d191b1bebc934a6ba4251a1055cf4bb3"
format_stamp: "Formatted at 2026-08-09 08:42:30 on dist-test-slave-gkw7"
server_key: "1ea9b1bfccb76a88893ae7aa514b4d4b"
server_key_iv: "2df51e3116b7d56fbe93b94f52be693c"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.316375 9220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:30.333920 9220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:30.335728 9220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:30.339057 9220 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.125:43031
I20260809 08:42:30.339309 9468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.125:43031 every 8 connection(s)
I20260809 08:42:30.339488 9220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
I20260809 08:42:30.340269 9469 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:42:30.340706 9469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: Bootstrap starting.
I20260809 08:42:30.343238 9469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:42:30.343480 9469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: Bootstrap complete.
I20260809 08:42:30.343730 9469 raft_consensus.cc:348] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } }
I20260809 08:42:30.343820 9469 raft_consensus.cc:729] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d191b1bebc934a6ba4251a1055cf4bb3, State: Initialized, Role: LEARNER
I20260809 08:42:30.343873 9469 consensus_queue.cc:260] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } }
I20260809 08:42:30.344087 9469 sys_catalog.cc:564] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:30.344089 9476 sys_catalog.cc:455] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } } }
I20260809 08:42:30.344264 9476 sys_catalog.cc:458] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: This master's current role is: LEARNER
I20260809 08:42:30.345615 9494 catalog_manager.cc:1259] Loaded cluster ID: aa58af3af45f4db4addcaa8215477801
I20260809 08:42:30.345661 9494 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: loading cluster ID for follower catalog manager: success
I20260809 08:42:30.346666 9494 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: acquiring CA information for follower catalog manager: success
I20260809 08:42:30.346943 9494 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:42:30.394402 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.386927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.31.63.125:43031)
0809 08:42:30.387064 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.387067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.387078 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:42:30.387108 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.387108 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.387118 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.387192 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.387939 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.388655 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.389781 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.390006 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.390066 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.391552 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.391554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.391554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.391562 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.393154 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.393614 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.393615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.393615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.393671 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.393923 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.393924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.393924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.394031 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.394108 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.394263 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.394330 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for oryx/127.31.63.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.124.keytab.
Entry for principal oryx/127.31.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.124.keytab.
I20260809 08:42:30.424337 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.124:38357
--webserver_interface=127.31.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.124.keytab
--principal=oryx/127.31.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33565,127.31.63.125:43031,127.31.63.124:38357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:30.511171 9499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:30.511404 9499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:30.511425 9499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:30.512501 9499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:42:30.512534 9499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:30.512548 9499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:30.512562 9499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:42:30.512573 9499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:42:30.514087 9499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33565,127.31.63.125:43031,127.31.63.124:38357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.124.keytab
--principal=oryx/127.31.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.124:38357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:30.514297 9499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:30.514467 9499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:30.519577 9511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:30.519580 9507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:30.519596 9514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:30.519855 9499 server_base.cc:1034] running on GCE node
I20260809 08:42:30.520102 9499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:30.520352 9499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:30.521515 9499 hybrid_clock.cc:648] HybridClock initialized: now 1786264950521503 us; error 25 us; skew 500 ppm
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:30.524130 9499 init.cc:377] Logged in from keytab as oryx/127.31.63.124@KRBTEST.COM (short username oryx)
I20260809 08:42:30.524700 9499 webserver.cc:460] Webserver started at http://127.31.63.124:45405/ using document root <none> and password file <none>
I20260809 08:42:30.524950 9499 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:30.525022 9499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:30.525184 9499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:42:30.526306 9499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data/instance:
uuid: "3cb1546ac394456396e1d828395c60a9"
format_stamp: "Formatted at 2026-08-09 08:42:30 on dist-test-slave-gkw7"
server_key: "5078617540395da6e5ce4a7ca537f819"
server_key_iv: "74f3a070288f1a36dc6d4a7b0326edb7"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.526729 9499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal/instance:
uuid: "3cb1546ac394456396e1d828395c60a9"
format_stamp: "Formatted at 2026-08-09 08:42:30 on dist-test-slave-gkw7"
server_key: "5078617540395da6e5ce4a7ca537f819"
server_key_iv: "74f3a070288f1a36dc6d4a7b0326edb7"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.528226 9499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:42:30.529054 9523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.529232 9499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:30.529315 9499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal
uuid: "3cb1546ac394456396e1d828395c60a9"
format_stamp: "Formatted at 2026-08-09 08:42:30 on dist-test-slave-gkw7"
server_key: "5078617540395da6e5ce4a7ca537f819"
server_key_iv: "74f3a070288f1a36dc6d4a7b0326edb7"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.529367 9499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:30.541128 9499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:30.544183 9499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:30.544296 9499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:30.550766 9499 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.124:38357
I20260809 08:42:30.550798 9597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.124:38357 every 8 connection(s)
I20260809 08:42:30.551162 9499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
I20260809 08:42:30.552305 9600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:30.553942 9600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.124@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:42:30.560031 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 9499
I20260809 08:42:30.563338 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.554790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54410 (local address 127.31.63.126:33565)
0809 08:42:30.554861 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.554863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.555033 (+ 170us) server_negotiation.cc:407] Connection header received
0809 08:42:30.555490 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.555491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.555502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.555564 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.556178 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.556724 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.557775 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.557961 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.558015 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.560159 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.560161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.560161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.560171 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.561971 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.562379 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.562380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.562381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.562441 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.562701 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.562703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.562703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.562822 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.562900 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.563166 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.563246 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:42:30.563607 9602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.554620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:54410)
0809 08:42:30.554903 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.554943 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.555405 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.555638 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.555641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.555878 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.556100 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.556103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.556806 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.556806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.557700 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.557703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.558057 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.558058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.558152 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.558663 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.558681 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.560038 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.562069 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.562077 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.562081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.562296 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.562518 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.562519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.562519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.562630 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.562892 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.562900 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.563026 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.563136 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.563296 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":192,"threads_started":1}
I20260809 08:42:30.564183 9600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.63.124@KRBTEST.COM
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.124@KRBTEST.COM for oryx/127.31.63.125@KRBTEST.COM
I20260809 08:42:30.570613 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.560756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33034 (local address 127.31.63.124:38357)
0809 08:42:30.560929 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.560933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.560959 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:42:30.560982 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.560984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.561003 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.561099 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.561942 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.562610 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.563497 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.563997 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.564072 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.567163 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.567171 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.567171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.567177 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.569121 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.569580 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.569584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.569584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.569649 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.569915 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.569916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.569916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.570108 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.570265 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.570376 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.570464 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260809 08:42:30.571441 9602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.564451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:43031 (local address 127.0.0.1:51332)
0809 08:42:30.564505 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.564515 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.564572 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.564710 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.564711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.564864 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.564922 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.564924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.565642 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.565644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.566190 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.566192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.566607 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.566608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.566684 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.567014 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.567019 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.568163 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.570020 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.570021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.570021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.570247 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.570503 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.570504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.570505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.570584 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.571168 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.571169 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.571215 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.571303 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.571387 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 08:42:30.571485 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.564519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.31.63.125:43031)
0809 08:42:30.564575 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.564577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.564586 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:42:30.564640 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.564641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.564652 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.564696 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.565002 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.565489 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.566289 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.566509 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.566607 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.568262 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.568263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.568263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.568273 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.569941 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.570349 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.570351 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.570351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.570418 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.570643 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.570645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.570645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.570788 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.571175 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.571346 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.571422 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:42:30.571970 9600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.124" port: 38357 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.124@KRBTEST.COM for oryx/127.31.63.124@KRBTEST.COM
I20260809 08:42:30.577903 9602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.572220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:38357 (local address 127.0.0.1:33046)
0809 08:42:30.572317 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.572325 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.572362 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.572458 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.572458 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.572563 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.572590 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.572591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.573174 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.573175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.573698 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.573699 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.574103 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.574104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.574141 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.574498 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.574502 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.575554 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.577086 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.577087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.577087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.577240 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.577408 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.577408 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.577409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.577458 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.577683 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.577684 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.577718 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.577771 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.577844 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 08:42:30.578029 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.572278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33046 (local address 127.31.63.124:38357)
0809 08:42:30.572307 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.572309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.572360 (+ 51us) server_negotiation.cc:407] Connection header received
0809 08:42:30.572396 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.572396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.572404 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.572448 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.572648 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.573109 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.573800 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.574032 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.574094 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.575628 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.575629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.575630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.575636 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.577017 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.577303 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.577303 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.577304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.577351 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.577494 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.577495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.577495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.577627 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.577677 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.577785 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.577889 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":15}
I20260809 08:42:30.579254 9600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: Bootstrap starting.
I20260809 08:42:30.580090 9600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:30.580399 9600 log.cc:824] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:30.581130 9600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: No bootstrap required, opened a new log
I20260809 08:42:30.582660 9600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 38357 } }
I20260809 08:42:30.582777 9600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:30.582799 9600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb1546ac394456396e1d828395c60a9, State: Initialized, Role: FOLLOWER
I20260809 08:42:30.582952 9600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 38357 } }
I20260809 08:42:30.583250 9600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:30.583225 9606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 38357 } } }
I20260809 08:42:30.583328 9606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:42:30.585265 9621 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:42:30.585382 9621 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:42:30.587591 9499 master_runner.cc:428] Detected that this master 3cb1546ac394456396e1d828395c60a9 is joining an existing cluster
I20260809 08:42:30.587658 9499 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.63.124:38357
I20260809 08:42:30.595656 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.588560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:54418)
0809 08:42:30.588749 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.588759 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.588832 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.588956 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.588957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.589150 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.589249 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.589252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.590151 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.590152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.590741 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.590743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.591209 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.591210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.591279 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.591749 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.591754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.592188 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.594142 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.594144 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.594144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.594383 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.594898 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.594899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.594899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.594996 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.595315 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.595316 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.595363 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.595430 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.595539 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
I20260809 08:42:30.596267 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.588564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54418 (local address 127.31.63.126:33565)
0809 08:42:30.588607 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.588609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.588819 (+ 210us) server_negotiation.cc:407] Connection header received
0809 08:42:30.588892 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.588893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.588904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.588954 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.589350 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.590068 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.590830 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.591134 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.591208 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.592280 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.592282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.592282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.592289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.594069 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.594748 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.594749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.594749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.594812 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.595084 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.595085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.595086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.595221 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.595285 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.596079 (+ 794us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.596158 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:42:30.596876 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.588868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51342 (local address 127.31.63.125:43031)
0809 08:42:30.588906 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.588908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.589292 (+ 384us) server_negotiation.cc:407] Connection header received
0809 08:42:30.589335 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.589336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.589347 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.589402 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.589847 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.590582 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.591624 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.591939 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.592015 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.593150 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.593151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.593152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.593159 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.595419 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.595784 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.595785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.595786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.595847 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.596036 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.596037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.596038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.596164 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.596235 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.596694 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.596773 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:42:30.596613 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.588943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:43031 (local address 127.0.0.1:51342)
0809 08:42:30.589196 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.589208 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.589278 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.589452 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.589453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.589668 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.589750 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.589752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.590684 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.590684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.591540 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.591544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.592056 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.592058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.592156 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.592675 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.592680 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.593074 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.595497 (+ 2423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.595498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.595498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.595694 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.595911 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.595912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.595912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.595976 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.596264 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.596265 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.596310 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.596390 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.596492 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":181,"threads_started":1}
I20260809 08:42:30.599701 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.589751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.31.63.124:38357)
0809 08:42:30.589786 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.589787 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.593736 (+ 3949us) server_negotiation.cc:407] Connection header received
0809 08:42:30.593761 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.593762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.593773 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.593830 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.594185 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.594845 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.595857 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.596149 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.596188 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.597226 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.597227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.597227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.597234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.598706 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.599072 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.599073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.599073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.599121 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.599314 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.599314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.599314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.599397 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.599443 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.599592 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.599651 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:42:30.599824 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.589815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:38357 (local address 127.0.0.1:33048)
0809 08:42:30.593643 (+ 3828us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.593653 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.593713 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.593833 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.593833 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.594034 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.594089 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.594091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.594931 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.594932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.595766 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.595768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.596255 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.596255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.596311 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.596698 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.596703 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.597145 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.598785 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.598785 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.598786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.598998 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.599185 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.599186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.599187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.599251 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.599459 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.599460 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.599502 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.599578 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.599717 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3811,"mutex_wait_us":45,"thread_start_us":104,"threads_started":1}
I20260809 08:42:30.622148 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 8655
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264949, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.126@KRBTEST.COM for oryx/127.31.63.124@KRBTEST.COM
I20260809 08:42:30.634647 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:34803
--local_ip_for_outbound_sockets=127.31.63.65
--tserver_master_addrs=127.31.63.126:33565
--webserver_port=41125
--webserver_interface=127.31.63.65
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab
--principal=oryx/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.63.126:33565,127.31.63.125:43031
--tserver_master_addrs=127.31.63.126:33565,127.31.63.125:43031,127.31.63.124:38357 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:42:30.636047 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.628795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.31.63.124:38357)
0809 08:42:30.628836 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.628838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.628847 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:42:30.628935 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.628937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.628950 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.629019 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.629370 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.629906 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.630605 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.630798 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.630876 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.632937 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.632939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.632939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.632947 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.634738 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.635157 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.635159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.635159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.635232 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.635465 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.635466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.635467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.635608 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.635678 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.635860 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.635949 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:42:30.636139 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.628732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:38357 (local address 127.0.0.1:33052)
0809 08:42:30.628794 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.628805 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.628852 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.629039 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.629040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.629226 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.629285 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.629288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.629988 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.629990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.630518 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.630520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.630874 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.630875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.630954 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.631379 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.631393 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.632843 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.634841 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.634842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.634842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.635064 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.635314 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.635316 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.635316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.635391 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.635683 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.635684 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.635741 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.635813 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.635938 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:42:30.636993 8568 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.63.124:38357: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3cb1546ac394456396e1d828395c60a9"
member_type: NON_VOTER
last_known_addr {
host: "127.31.63.124"
port: 38357
}
attrs {
promote: true
}
}
dest_uuid: "021edbb7538f4488976e2bbed5f7f747"
cas_config_opid_index: 5
I20260809 08:42:30.637274 8568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } }
I20260809 08:42:30.637666 9352 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3cb1546ac394456396e1d828395c60a9 (127.31.63.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.638883 9352 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.63.124:38357
I20260809 08:42:30.638967 9352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "021edbb7538f4488976e2bbed5f7f747" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.639042 9352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:30.639133 9499 master.cc:537] Master@127.31.63.124:38357 shutting down...
W20260809 08:42:30.639570 8541 consensus_peers.cc:487] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 -> Peer 3cb1546ac394456396e1d828395c60a9 (127.31.63.124:38357): Couldn't send request to peer 3cb1546ac394456396e1d828395c60a9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 08:42:30.641433 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.637726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:43031 (local address 127.0.0.1:51354)
0809 08:42:30.637766 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.637776 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.637811 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.638054 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.638055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.638057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:30.638096 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.638097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.639029 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.639030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.640244 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.640247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.641165 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.641166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.641239 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.641263 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.641325 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.641363 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 08:42:30.641702 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.637818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.31.63.125:43031)
0809 08:42:30.637860 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.637862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.637870 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:30.637897 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.637898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.637925 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.637994 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:30.638184 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.638933 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.640353 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.641063 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.641151 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.641257 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.641585 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.641634 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:42:30.641963 9434 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 LEARNER]: Refusing update from remote peer 021edbb7538f4488976e2bbed5f7f747: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 08:42:30.642210 9355 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 08:42:30.642951 9434 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3cb1546ac394456396e1d828395c60a9 (127.31.63.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.644232 9635 sys_catalog.cc:455] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "021edbb7538f4488976e2bbed5f7f747" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.644457 9635 sys_catalog.cc:458] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: This master's current role is: LEARNER
I20260809 08:42:30.644771 9499 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:30.644761 9476 sys_catalog.cc:455] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 021edbb7538f4488976e2bbed5f7f747. Latest consensus state: current_term: 1 leader_uuid: "021edbb7538f4488976e2bbed5f7f747" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: NON_VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: true } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.644856 9476 sys_catalog.cc:458] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: This master's current role is: LEARNER
I20260809 08:42:30.644865 9499 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:30.644917 9499 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: stopping tablet replica
I20260809 08:42:30.646442 9355 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747: attempting to promote NON_VOTER d191b1bebc934a6ba4251a1055cf4bb3 to VOTER
I20260809 08:42:30.646631 9355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } }
W20260809 08:42:30.647408 8541 connection.cc:535] client connection to 127.31.63.124:38357 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 08:42:30.647468 8541 consensus_peers.cc:487] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 -> Peer 3cb1546ac394456396e1d828395c60a9 (127.31.63.124:38357): Couldn't send request to peer 3cb1546ac394456396e1d828395c60a9. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260809 08:42:30.648370 9434 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 LEARNER]: Refusing update from remote peer 021edbb7538f4488976e2bbed5f7f747: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 08:42:30.648624 9355 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 08:42:30.649300 9355 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d191b1bebc934a6ba4251a1055cf4bb3 (127.31.63.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.649454 9434 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d191b1bebc934a6ba4251a1055cf4bb3 (127.31.63.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.650517 9355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "021edbb7538f4488976e2bbed5f7f747" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.650610 9355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:30.650661 9355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "021edbb7538f4488976e2bbed5f7f747" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.650700 9355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747 [sys.catalog]: This master's current role is: LEADER
I20260809 08:42:30.650842 9476 sys_catalog.cc:455] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "021edbb7538f4488976e2bbed5f7f747" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.650949 9476 sys_catalog.cc:458] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:42:30.659427 9499 master.cc:552] Master@127.31.63.124:38357 shutdown complete.
I20260809 08:42:30.659487 9499 master_runner.cc:305] Clearing existing system tablet
I20260809 08:42:30.659636 9499 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:30.659685 9499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:30.660444 9499 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:42:30.661099 9640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.661276 9499 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:30.661342 9499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:30.661371 9499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal
uuid: "3cb1546ac394456396e1d828395c60a9"
format_stamp: "Formatted at 2026-08-09 08:42:30 on dist-test-slave-gkw7"
server_key: "5078617540395da6e5ce4a7ca537f819"
server_key_iv: "74f3a070288f1a36dc6d4a7b0326edb7"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.662210 9499 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:30.663687 9499 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 08:42:30.663761 9499 log.cc:1196] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 08:42:30.663954 9499 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: Deleting consensus metadata
I20260809 08:42:30.664255 9499 master_runner.cc:315] Copying system tablet from 127.31.63.126:33565
I20260809 08:42:30.664913 9499 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: tablet copy: Beginning tablet copy session from remote peer at address 127.31.63.126:33565
I20260809 08:42:30.672329 9649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.665218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:54426)
0809 08:42:30.665348 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.665360 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.665428 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.665654 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.665655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.665865 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.665923 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.665926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.666893 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.666894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.667740 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.667743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.668228 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.668229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.668305 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.668861 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.668866 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.669296 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.671190 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.671191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.671191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.671413 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.671632 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.671632 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.671633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.671700 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.671984 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.671985 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.672033 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.672119 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.672227 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260809 08:42:30.672330 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.665289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.31.63.126:33565)
0809 08:42:30.665337 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.665340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.665413 (+ 73us) server_negotiation.cc:407] Connection header received
0809 08:42:30.665485 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.665486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.665498 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.665644 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.666015 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.666792 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.667837 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.668134 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.668209 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.669396 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.669398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.669398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.669406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.671098 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.671490 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.671491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.671492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.671556 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.671766 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.671767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.671767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.671898 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.671979 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.672159 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.672241 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:42:30.672938 8593 tablet_copy_service.cc:140] P 021edbb7538f4488976e2bbed5f7f747: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3cb1546ac394456396e1d828395c60a9 ({username='oryx', principal='oryx/127.31.63.124@KRBTEST.COM'} at 127.0.0.1:54426)
I20260809 08:42:30.673002 8593 tablet_copy_service.cc:161] P 021edbb7538f4488976e2bbed5f7f747: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3cb1546ac394456396e1d828395c60a9 at {username='oryx', principal='oryx/127.31.63.124@KRBTEST.COM'} at 127.0.0.1:54426: session id = 3cb1546ac394456396e1d828395c60a9-00000000000000000000000000000000
I20260809 08:42:30.673638 8593 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 08:42:30.674109 9499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 08:42:30.675498 9499 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: tablet copy: Starting download of 0 data blocks...
I20260809 08:42:30.675657 9499 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: tablet copy: Starting download of 1 WAL segments...
I20260809 08:42:30.676420 9499 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 08:42:30.677276 8593 tablet_copy_service.cc:342] P 021edbb7538f4488976e2bbed5f7f747: Request end of tablet copy session 3cb1546ac394456396e1d828395c60a9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.63.124@KRBTEST.COM'} at 127.0.0.1:54426
I20260809 08:42:30.677335 8593 tablet_copy_service.cc:433] P 021edbb7538f4488976e2bbed5f7f747: ending tablet copy session 3cb1546ac394456396e1d828395c60a9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3cb1546ac394456396e1d828395c60a9
I20260809 08:42:30.679879 9499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:30.681041 9651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:30.681473 9656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:30.685659 9652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:30.686852 9499 server_base.cc:1034] running on GCE node
I20260809 08:42:30.687013 9499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:30.687201 9499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:30.690446 9499 hybrid_clock.cc:648] HybridClock initialized: now 1786264950690425 us; error 43 us; skew 500 ppm
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:30.692034 9499 init.cc:377] Logged in from keytab as oryx/127.31.63.124@KRBTEST.COM (short username oryx)
I20260809 08:42:30.692333 9499 webserver.cc:460] Webserver started at http://127.31.63.124:43191/ using document root <none> and password file <none>
I20260809 08:42:30.692461 9499 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:30.692502 9499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:30.693291 9499 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:30.693987 9664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.694167 9499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.694227 9499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal
uuid: "3cb1546ac394456396e1d828395c60a9"
format_stamp: "Formatted at 2026-08-09 08:42:30 on dist-test-slave-gkw7"
server_key: "5078617540395da6e5ce4a7ca537f819"
server_key_iv: "74f3a070288f1a36dc6d4a7b0326edb7"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.694291 9499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:30.708750 9499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:30.710733 9499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:30.714108 9499 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.124:38357
I20260809 08:42:30.714397 9738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.124:38357 every 8 connection(s)
I20260809 08:42:30.714571 9499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
I20260809 08:42:30.715500 9739 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:42:30.716050 9739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: Bootstrap starting.
I20260809 08:42:30.719111 9739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:42:30.719347 9739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: Bootstrap complete.
I20260809 08:42:30.719599 9739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } }
I20260809 08:42:30.719726 9739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3cb1546ac394456396e1d828395c60a9, State: Initialized, Role: LEARNER
I20260809 08:42:30.719770 9739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } }
I20260809 08:42:30.719992 9739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:42:30.721443 9742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } } }
I20260809 08:42:30.721980 9742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9 [sys.catalog]: This master's current role is: LEARNER
I20260809 08:42:30.721751 9758 catalog_manager.cc:1259] Loaded cluster ID: aa58af3af45f4db4addcaa8215477801
I20260809 08:42:30.722751 9758 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: loading cluster ID for follower catalog manager: success
I20260809 08:42:30.723809 9758 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: acquiring CA information for follower catalog manager: success
I20260809 08:42:30.724112 9758 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3cb1546ac394456396e1d828395c60a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 08:42:30.741725 9632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:30.741963 9632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:30.742002 9632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:30.743382 9632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:30.743458 9632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:30.743505 9632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:42:30.745281 9632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab
--principal=oryx/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:34803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=41125
--tserver_master_addrs=127.31.63.126:33565,127.31.63.125:43031,127.31.63.124:38357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:30.745615 9632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:30.745857 9632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:30.751384 9773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:30.751361 9767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:30.751402 9632 server_base.cc:1034] running on GCE node
W20260809 08:42:30.751565 9770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:30.751927 9632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:30.752202 9632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:30.753396 9632 hybrid_clock.cc:648] HybridClock initialized: now 1786264950753366 us; error 45 us; skew 500 ppm
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:30.756956 9632 init.cc:377] Logged in from keytab as oryx/127.31.63.65@KRBTEST.COM (short username oryx)
I20260809 08:42:30.757474 9632 webserver.cc:460] Webserver started at http://127.31.63.65:41125/ using document root <none> and password file <none>
I20260809 08:42:30.757766 9632 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:30.757853 9632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:30.759445 9632 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:30.760279 9783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.760509 9632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:30.760630 9632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "f4009453748648e8aa37ea67947bcb00"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "7aa1823581771d12c4f89e5382802fe9"
server_key_iv: "34dd34355083a38d52069c832573781c"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.761034 9632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:30.788067 9632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:30.790858 9632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:30.791036 9632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:30.791453 9632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:30.791795 9632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:30.791849 9632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.791893 9632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:30.791917 9632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.804471 9632 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:34803
I20260809 08:42:30.804579 9936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:34803 every 8 connection(s)
I20260809 08:42:30.804936 9632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:42:30.813232 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 9632
I20260809 08:42:30.813423 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 8843
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.125@KRBTEST.COM
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.124@KRBTEST.COM
I20260809 08:42:30.820132 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.806062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:37125 (local address 127.31.63.125:43031)
0809 08:42:30.806130 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.806132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.806704 (+ 572us) server_negotiation.cc:407] Connection header received
0809 08:42:30.807131 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.807132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.807144 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.807194 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.808207 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.808999 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.810614 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.810948 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.811023 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.816509 (+ 5486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.816511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.816511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.816519 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.818383 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.819122 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.819124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.819124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.819183 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.819420 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.819421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.819421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.819553 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.819622 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.819948 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.820032 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:42:30.820297 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.806029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:49141 (local address 127.31.63.126:33565)
0809 08:42:30.806408 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.806410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.806419 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:42:30.807188 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.807191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.807209 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.807255 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.808259 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.808998 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.810616 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.810825 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.810859 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.816740 (+ 5881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.816741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.816742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.816750 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.818517 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.819005 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.819007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.819008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.819068 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.819373 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.819375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.819375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.819518 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.819591 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.819997 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.820121 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344}
I20260809 08:42:30.820761 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.805953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:43031 (local address 127.31.63.65:37125)
0809 08:42:30.806619 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.806630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.807035 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.807596 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.807597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.807835 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.808114 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.808120 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.809180 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.809181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.810492 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.810495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.811075 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.811076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.811161 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.812044 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.812052 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.816387 (+ 4335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.818488 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.818497 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.818500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.818970 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.819263 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.819264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.819264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.819352 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.819652 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.819653 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.819791 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.819913 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.820086 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":670,"thread_start_us":118,"threads_started":1}
I20260809 08:42:30.820785 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.805954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:49141)
0809 08:42:30.806328 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.806339 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.807028 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.807370 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.807375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.807892 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.808115 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.808123 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.809136 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.809137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.810492 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.810495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.810934 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.810935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.811128 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.811982 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.811994 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.816646 (+ 4652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.818617 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.818618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.818619 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.818927 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.819167 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.819167 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.819168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.819299 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.819598 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.819607 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.819793 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.820412 (+ 619us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.820552 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":185,"thread_start_us":106,"threads_started":1}
I20260809 08:42:30.821683 9939 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33565
I20260809 08:42:30.821825 9939 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:30.822072 9939 heartbeater.cc:511] Master 127.31.63.126:33565 requested a full tablet report, sending...
I20260809 08:42:30.822943 8568 ts_manager.cc:194] Re-registered known tserver with Master: f4009453748648e8aa37ea67947bcb00 (127.31.63.65:34803)
I20260809 08:42:30.823498 8568 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:49141
I20260809 08:42:30.823915 9941 heartbeater.cc:347] Connected to a master server at 127.31.63.125:43031
I20260809 08:42:30.823987 9941 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:30.824388 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:41417
--local_ip_for_outbound_sockets=127.31.63.66
--tserver_master_addrs=127.31.63.126:33565
--webserver_port=46411
--webserver_interface=127.31.63.66
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab
--principal=oryx/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.63.126:33565,127.31.63.125:43031
--tserver_master_addrs=127.31.63.126:33565,127.31.63.125:43031,127.31.63.124:38357 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:42:30.825248 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.805953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:38357 (local address 127.31.63.65:49763)
0809 08:42:30.806190 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.806243 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.807028 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.807370 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.807375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.807890 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.808115 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.808123 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.809163 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.809164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.810407 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.810410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.810985 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.810986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.811156 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.811982 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.811994 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.816526 (+ 4532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.824106 (+ 7580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.824107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.824108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.824286 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.824523 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.824524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.824524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.824602 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.824888 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.824888 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.824931 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.825033 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.825135 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":93,"thread_start_us":158,"threads_started":1}
I20260809 08:42:30.825275 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.806267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:49763 (local address 127.31.63.124:38357)
0809 08:42:30.806404 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.806406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.806419 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:42:30.807190 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.807192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.807209 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.807268 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.808291 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.809078 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.810549 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.810899 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.810986 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.822108 (+ 11122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.822110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.822110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.822119 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.824004 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.824384 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.824386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.824386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.824445 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.824677 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.824678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.824679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.824804 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.824895 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.825082 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.825173 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260809 08:42:30.825619 9938 heartbeater.cc:347] Connected to a master server at 127.31.63.124:38357
I20260809 08:42:30.825668 9938 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:30.825726 9938 heartbeater.cc:511] Master 127.31.63.124:38357 requested a full tablet report, sending...
I20260809 08:42:30.826088 9676 ts_manager.cc:194] Registered new tserver with Master: f4009453748648e8aa37ea67947bcb00 (127.31.63.65:34803)
I20260809 08:42:30.826671 9941 heartbeater.cc:511] Master 127.31.63.125:43031 requested a full tablet report, sending...
I20260809 08:42:30.827190 9418 ts_manager.cc:194] Registered new tserver with Master: f4009453748648e8aa37ea67947bcb00 (127.31.63.65:34803)
W20260809 08:42:30.909549 9948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:30.909757 9948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:30.909781 9948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:30.910931 9948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:30.910974 9948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:30.911000 9948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:42:30.912406 9948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab
--principal=oryx/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:41417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=46411
--tserver_master_addrs=127.31.63.126:33565,127.31.63.125:43031,127.31.63.124:38357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:30.912580 9948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:30.912745 9948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:42:30.917954 9964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:30.918008 9962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:30.918133 9948 server_base.cc:1034] running on GCE node
W20260809 08:42:30.917979 9961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:30.918382 9948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:30.918634 9948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:30.919806 9948 hybrid_clock.cc:648] HybridClock initialized: now 1786264950919794 us; error 46 us; skew 500 ppm
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:30.922487 9948 init.cc:377] Logged in from keytab as oryx/127.31.63.66@KRBTEST.COM (short username oryx)
I20260809 08:42:30.922899 9948 webserver.cc:460] Webserver started at http://127.31.63.66:46411/ using document root <none> and password file <none>
I20260809 08:42:30.923152 9948 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:30.923242 9948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:30.924573 9948 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:30.925252 9977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.925443 9948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:42:30.925591 9948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "a3e27e0401e54d88a1217cbe1ff301e8"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "0c3f6961c1474fd71aadb2755c0141c6"
server_key_iv: "f48dd97927302169a661bfbc35dba38a"
server_key_version: "encryptionkey@0"
I20260809 08:42:30.925951 9948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:30.934952 9948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:30.937155 9948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:30.937350 9948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:30.937690 9948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:30.938060 9948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:30.938129 9948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.938176 9948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:30.938210 9948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:30.946108 9948 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:41417
I20260809 08:42:30.946187 10138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:41417 every 8 connection(s)
I20260809 08:42:30.946602 9948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:42:30.950531 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 9948
I20260809 08:42:30.950732 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 9034
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.124@KRBTEST.COM
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:42:30.963195 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.947802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:40787 (local address 127.31.63.126:33565)
0809 08:42:30.947857 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.947859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.948176 (+ 317us) server_negotiation.cc:407] Connection header received
0809 08:42:30.949373 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.949374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.949390 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.949475 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.950383 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.951121 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.952604 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.952891 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.952933 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.959454 (+ 6521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.959455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.959456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.959464 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.961185 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.962037 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.962039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.962039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.962110 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.962389 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.962390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.962391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.962539 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.962591 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.962939 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.963030 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:42:30.963852 10145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.947431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:40787)
0809 08:42:30.948096 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.948107 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.949157 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.949483 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.949484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.950050 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.950275 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.950280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.951251 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.951253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.952496 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.952499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.953045 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.953046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.953183 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.954189 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.954196 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.959350 (+ 5154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.961283 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.961291 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.961294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.961949 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.962181 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.962182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.962182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.962314 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.962622 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.962627 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.962796 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.963324 (+ 528us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.963521 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":208,"thread_start_us":75,"threads_started":1}
I20260809 08:42:30.964299 10140 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33565
I20260809 08:42:30.964395 10140 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:30.964600 10140 heartbeater.cc:511] Master 127.31.63.126:33565 requested a full tablet report, sending...
I20260809 08:42:30.965058 8568 ts_manager.cc:194] Re-registered known tserver with Master: a3e27e0401e54d88a1217cbe1ff301e8 (127.31.63.66:41417)
I20260809 08:42:30.965668 8568 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:40787
I20260809 08:42:30.966861 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.947806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:36997 (local address 127.31.63.124:38357)
0809 08:42:30.947946 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.947948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.947959 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:42:30.949282 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.949284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.949305 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.949382 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.950425 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.951414 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.953072 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.953454 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.953505 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.956462 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.956464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.956464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.956474 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.965460 (+ 8986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.965952 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.965953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.965954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.966024 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.966255 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.966256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.966256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.966411 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.966472 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.966706 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.966794 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260809 08:42:30.967154 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.947431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:38357 (local address 127.31.63.66:36997)
0809 08:42:30.947689 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.947740 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.949146 (+ 1406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.949480 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.949484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.950102 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.950277 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.950284 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.951561 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.951562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.952903 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.952907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.953622 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.953623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.953696 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.954175 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.954186 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.956353 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.965589 (+ 9236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.965590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.965590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.965850 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.966109 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.966110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.966110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.966175 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.966511 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.966512 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.966553 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.966968 (+ 415us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.967074 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":12,"thread_start_us":113,"threads_started":1}
I20260809 08:42:30.967527 10139 heartbeater.cc:347] Connected to a master server at 127.31.63.124:38357
I20260809 08:42:30.967574 10139 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:30.967648 10139 heartbeater.cc:511] Master 127.31.63.124:38357 requested a full tablet report, sending...
I20260809 08:42:30.967919 9676 ts_manager.cc:194] Registered new tserver with Master: a3e27e0401e54d88a1217cbe1ff301e8 (127.31.63.66:41417)
Aug 09 08:42:30 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264950, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.125@KRBTEST.COM
I20260809 08:42:30.975238 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.948011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:43619 (local address 127.31.63.125:43031)
0809 08:42:30.948107 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:30.948109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:30.948121 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:42:30.949540 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:30.949541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:30.949589 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:30.949642 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:30.967356 (+ 17714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.968164 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.968921 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.969296 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.969364 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.972255 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:30.972257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:30.972258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:30.972267 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:30.973891 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.974389 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.974390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.974391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.974452 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:30.974701 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:30.974702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:30.974703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:30.974817 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:30.974872 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:30.975069 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 08:42:30.975168 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 08:42:30.975761 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:30.947446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:43031 (local address 127.31.63.66:43619)
0809 08:42:30.947769 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:30.947783 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:30.949145 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:30.949669 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:30.949670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:30.950084 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:30.950277 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.950284 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.968250 (+ 17966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.968251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.968826 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:30.968827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:30.969424 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:30.969425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:30.969511 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:30.970173 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:30.970180 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:30.972131 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:30.973990 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.973991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.973992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.974284 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.974537 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:30.974538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:30.974538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:30.974626 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:30.974889 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:30.974890 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:30.974948 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 08:42:30.975047 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 08:42:30.975200 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":35,"thread_start_us":147,"threads_started":1}
I20260809 08:42:30.976542 10141 heartbeater.cc:347] Connected to a master server at 127.31.63.125:43031
I20260809 08:42:30.976639 10141 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:30.976711 10141 heartbeater.cc:511] Master 127.31.63.125:43031 requested a full tablet report, sending...
I20260809 08:42:30.977036 9418 ts_manager.cc:194] Registered new tserver with Master: a3e27e0401e54d88a1217cbe1ff301e8 (127.31.63.66:41417)
I20260809 08:42:30.977293 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:36907
--local_ip_for_outbound_sockets=127.31.63.67
--tserver_master_addrs=127.31.63.126:33565
--webserver_port=34425
--webserver_interface=127.31.63.67
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab
--principal=oryx/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.63.126:33565,127.31.63.125:43031
--tserver_master_addrs=127.31.63.126:33565,127.31.63.125:43031,127.31.63.124:38357 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:42:31.076505 10147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:42:31.076763 10147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:42:31.076794 10147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:42:31.078112 10147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:42:31.078167 10147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:42:31.078198 10147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:42:31.079725 10147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab
--principal=oryx/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:36907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=34425
--tserver_master_addrs=127.31.63.126:33565,127.31.63.125:43031,127.31.63.124:38357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:42:31.079931 10147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:42:31.080125 10147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:42:31.085738 10147 server_base.cc:1034] running on GCE node
W20260809 08:42:31.085697 10159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:31.085697 10156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:42:31.085706 10155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:42:31.086156 10147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:42:31.086404 10147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:42:31.087582 10147 hybrid_clock.cc:648] HybridClock initialized: now 1786264951087569 us; error 29 us; skew 500 ppm
Aug 09 08:42:31 dist-test-slave-gkw7 krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264951, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:42:31.090415 10147 init.cc:377] Logged in from keytab as oryx/127.31.63.67@KRBTEST.COM (short username oryx)
I20260809 08:42:31.090909 10147 webserver.cc:460] Webserver started at http://127.31.63.67:34425/ using document root <none> and password file <none>
I20260809 08:42:31.091140 10147 fs_manager.cc:359] Metadata directory not provided
I20260809 08:42:31.091202 10147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:42:31.092598 10147 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:31.093257 10170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:31.093433 10147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:42:31.093519 10147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0"
format_stamp: "Formatted at 2026-08-09 08:42:29 on dist-test-slave-gkw7"
server_key: "1373576645147ec92b97395e3f52b5f2"
server_key_iv: "e5b888f5937e079e1356e41720203963"
server_key_version: "encryptionkey@0"
I20260809 08:42:31.093950 10147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:42:31.117666 10147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:42:31.120464 10147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:42:31.120637 10147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:42:31.120889 10147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:42:31.121276 10147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:42:31.121332 10147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:31.121371 10147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:42:31.121402 10147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:42:31.128074 10147 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:36907
I20260809 08:42:31.128156 10331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:36907 every 8 connection(s)
I20260809 08:42:31.128610 10147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:42:31.133741 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 10147
Aug 09 08:42:31 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264951, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
Aug 09 08:42:31 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264951, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.125@KRBTEST.COM
Aug 09 08:42:31 dist-test-slave-gkw7 krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264951, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.124@KRBTEST.COM
I20260809 08:42:31.141841 10343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.130034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:43031 (local address 127.31.63.67:56119)
0809 08:42:31.130174 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:31.130191 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:31.130553 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:31.131280 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:31.131281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:31.131427 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:31.131500 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.131502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.132431 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.132431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.133365 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.133368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.133937 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.133937 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.134013 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.134483 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:31.134487 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:31.136873 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:31.139431 (+ 2558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:31.139432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:31.139433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:31.139819 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:31.140511 (+ 692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:31.140512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:31.140513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:31.140684 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:31.141013 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:31.141021 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:31.141198 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 08:42:31.141320 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 08:42:31.141512 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":56,"thread_start_us":70,"threads_started":1}
I20260809 08:42:31.142424 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.130377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:56119 (local address 127.31.63.125:43031)
0809 08:42:31.130451 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.130453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.130463 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:31.130709 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.130710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.130721 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.130754 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:31.131596 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.132330 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.133469 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.133819 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.133857 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.136975 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:31.136976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:31.136977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:31.136985 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:31.139349 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.139904 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.139906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.139906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.140424 (+ 518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.140763 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.140764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.140764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.140916 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:31.140982 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.142240 (+ 1258us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.142326 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:42:31.142791 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.129440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:33779 (local address 127.31.63.126:33565)
0809 08:42:31.129495 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.129498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.129784 (+ 286us) server_negotiation.cc:407] Connection header received
0809 08:42:31.130684 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.130686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.130701 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.130734 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:31.131526 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.132076 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.133138 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.133363 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.133444 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.139707 (+ 6263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:31.139709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:31.139709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:31.139718 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:31.141464 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.141889 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.141890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.141890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.141952 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.142184 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.142185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.142185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.142312 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:31.142365 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.142563 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.142674 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:42:31.142813 10336 heartbeater.cc:347] Connected to a master server at 127.31.63.125:43031
I20260809 08:42:31.142913 10336 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:31.143108 10342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.129384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:33779)
0809 08:42:31.129656 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:31.129692 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:31.130543 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:31.130894 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:31.130897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:31.131218 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:31.131429 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.131433 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.132200 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.132201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.133030 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.133032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.133452 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.133454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.133647 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.134232 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:31.134244 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:31.138504 (+ 4260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:31.141585 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:31.141586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:31.141587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:31.141779 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:31.142037 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:31.142038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:31.142038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:31.142110 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:31.142399 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:31.142400 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:31.142442 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 08:42:31.142865 (+ 423us) client_negotiation.cc:240] Negotiation successful
0809 08:42:31.142984 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":115,"thread_start_us":105,"threads_started":1}
I20260809 08:42:31.143124 10336 heartbeater.cc:511] Master 127.31.63.125:43031 requested a full tablet report, sending...
I20260809 08:42:31.143610 9418 ts_manager.cc:194] Registered new tserver with Master: 0d14dbfb74a142e5a9e5b9552ccabfa0 (127.31.63.67:36907)
I20260809 08:42:31.143987 10333 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33565
I20260809 08:42:31.144048 10333 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:31.144129 10333 heartbeater.cc:511] Master 127.31.63.126:33565 requested a full tablet report, sending...
I20260809 08:42:31.144361 8568 ts_manager.cc:194] Re-registered known tserver with Master: 0d14dbfb74a142e5a9e5b9552ccabfa0 (127.31.63.67:36907)
I20260809 08:42:31.144654 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.129419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:40341 (local address 127.31.63.124:38357)
0809 08:42:31.129480 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.129483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.129784 (+ 301us) server_negotiation.cc:407] Connection header received
0809 08:42:31.130668 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.130670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.130684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.130717 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:31.131546 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.132331 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.133469 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.133805 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.133851 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.137483 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:31.137485 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:31.137485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:31.137495 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:31.139251 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.143688 (+ 4437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.143689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.143689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.143759 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.144019 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.144020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.144020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.144177 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:31.144236 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.144463 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.144561 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:42:31.144816 10354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.136321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54442 (local address 127.31.63.126:33565)
0809 08:42:31.136676 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.136678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.136814 (+ 136us) server_negotiation.cc:407] Connection header received
0809 08:42:31.136868 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.136869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.136882 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.136951 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:31.137672 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.138415 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.139473 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.139769 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.139825 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.141350 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:31.141352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:31.141352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:31.141359 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:31.143311 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.143745 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.143746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.143747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.143811 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.144108 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.144109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.144109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.144254 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:31.144353 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.144609 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.144697 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":85,"threads_started":1}
I20260809 08:42:31.144836 8568 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:33779
I20260809 08:42:31.145066 10340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.129387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:38357 (local address 127.31.63.67:40341)
0809 08:42:31.129636 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:31.129700 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:31.130547 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:31.130915 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:31.130916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:31.131240 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:42:31.131430 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.131435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.132431 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.132431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.133376 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.133378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.133937 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.133937 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.134013 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.134553 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:42:31.134557 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:42:31.137373 (+ 2816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:42:31.139353 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:31.139362 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:31.139364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:31.143617 (+ 4253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:31.143868 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:42:31.143869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:42:31.143869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:42:31.143938 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:42:31.144288 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:42:31.144289 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:42:31.144334 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 08:42:31.144407 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:42:31.144561 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":31,"thread_start_us":96,"threads_started":1}
I20260809 08:42:31.145756 10332 heartbeater.cc:347] Connected to a master server at 127.31.63.124:38357
I20260809 08:42:31.145812 10332 heartbeater.cc:464] Registering TS with master...
I20260809 08:42:31.145882 10332 heartbeater.cc:511] Master 127.31.63.124:38357 requested a full tablet report, sending...
I20260809 08:42:31.146162 9676 ts_manager.cc:194] Registered new tserver with Master: 0d14dbfb74a142e5a9e5b9552ccabfa0 (127.31.63.67:36907)
I20260809 08:42:31.147482 8568 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:42:31.149930 8568 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:42:31.150491 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.135996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.31.63.125:43031)
0809 08:42:31.136135 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.136137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.136145 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:31.136168 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.136169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.136193 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.136637 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:31.137581 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.138356 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.142129 (+ 3773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.142469 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.145782 (+ 3313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.147231 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:31.147232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:31.147233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:31.147240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:31.149211 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.149652 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.149653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.149653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.149711 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.149926 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.149927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.149927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.150069 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:31.150186 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.150293 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.150384 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260809 08:42:31.151131 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.135709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.31.63.124:38357)
0809 08:42:31.135934 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.135936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.136220 (+ 284us) server_negotiation.cc:407] Connection header received
0809 08:42:31.136318 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.136319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.136332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.136371 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:31.139079 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.139813 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.146007 (+ 6194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.146295 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.146360 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.147792 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:31.147794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:31.147794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:31.147802 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:31.149474 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.149916 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.149917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.149918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.149977 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.150182 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.150183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.150184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.150324 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:31.150396 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.150978 (+ 582us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.151066 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":79,"threads_started":1}
I20260809 08:42:31.158252 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.154131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:34803 (local address 127.0.0.1:59672)
0809 08:42:31.154370 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:31.154383 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:31.154433 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:31.155310 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:31.155311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:31.155314 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:31.155357 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.155358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.156200 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.156201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.157058 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.157060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.157983 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.157984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.158053 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.158059 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:42:31.158120 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:42:31.158163 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":77,"threads_started":1}
I20260809 08:42:31.158259 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.154445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59672 (local address 127.31.63.65:34803)
0809 08:42:31.154860 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.154863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.154872 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:42:31.154905 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.154907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.154919 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.155312 (+ 393us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:31.155417 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.156110 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.157145 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.157887 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.157979 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.158064 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.158136 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.158174 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":101,"threads_started":1}
I20260809 08:42:31.158900 10368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.154077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:41417 (local address 127.0.0.1:53972)
0809 08:42:31.154250 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:31.154261 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:31.154318 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:31.155221 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:31.155222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:31.155224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:31.155265 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.155267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.156485 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.156486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.157707 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.157710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.158645 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.158647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.158731 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.158735 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:42:31.158795 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:42:31.158836 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":64,"threads_started":1}
I20260809 08:42:31.158955 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.154417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53972 (local address 127.31.63.66:41417)
0809 08:42:31.155059 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.155063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.155073 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:31.155112 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.155114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.155131 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.155208 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:31.155357 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.156391 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.157802 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.158553 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.158632 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.158717 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.158822 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.158868 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":99,"threads_started":1}
I20260809 08:42:31.159077 10367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.154250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:36907 (local address 127.0.0.1:33926)
0809 08:42:31.154505 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:31.154513 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:31.154548 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:31.154990 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:31.154991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:31.154993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:31.155047 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.155049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.156682 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.156682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.157850 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.157852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.158859 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.158860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.158923 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.158927 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:42:31.158959 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 08:42:31.159002 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":13,"thread_start_us":75,"threads_started":1}
I20260809 08:42:31.159283 9846 tablet_service.cc:1467] Processing CreateTablet for tablet a4c1964a0e5146109ecc3c71c0a1f46b (DEFAULT_TABLE table=test-table [id=3ed7b38915284a90a9302d63dba122f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:31.159417 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.154677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.31.63.67:36907)
0809 08:42:31.154834 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.154838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.154846 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:31.154878 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.154880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.154893 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.154960 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:31.155113 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.156249 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.157942 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.158762 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.159186 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.159276 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.159302 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.159343 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260809 08:42:31.159662 9846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c1964a0e5146109ecc3c71c0a1f46b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:31.159577 10044 tablet_service.cc:1467] Processing CreateTablet for tablet a4c1964a0e5146109ecc3c71c0a1f46b (DEFAULT_TABLE table=test-table [id=3ed7b38915284a90a9302d63dba122f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:31.159914 10044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c1964a0e5146109ecc3c71c0a1f46b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:31.160064 10239 tablet_service.cc:1467] Processing CreateTablet for tablet a4c1964a0e5146109ecc3c71c0a1f46b (DEFAULT_TABLE table=test-table [id=3ed7b38915284a90a9302d63dba122f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:42:31.160383 10239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c1964a0e5146109ecc3c71c0a1f46b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:42:31.162822 10377 tablet_bootstrap.cc:492] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: Bootstrap starting.
I20260809 08:42:31.163479 10377 tablet_bootstrap.cc:654] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:31.163658 10379 tablet_bootstrap.cc:492] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: Bootstrap starting.
I20260809 08:42:31.164220 10377 log.cc:824] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:31.164460 10379 tablet_bootstrap.cc:654] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:31.164588 10381 tablet_bootstrap.cc:492] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: Bootstrap starting.
I20260809 08:42:31.165010 10379 log.cc:824] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:31.165148 10377 tablet_bootstrap.cc:492] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: No bootstrap required, opened a new log
I20260809 08:42:31.165211 10377 ts_tablet_manager.cc:1397] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:31.165455 10381 tablet_bootstrap.cc:654] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: Neither blocks nor log segments found. Creating new log.
I20260809 08:42:31.165817 10381 log.cc:824] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:42:31.165907 10379 tablet_bootstrap.cc:492] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: No bootstrap required, opened a new log
I20260809 08:42:31.165973 10379 ts_tablet_manager.cc:1397] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:31.166682 10381 tablet_bootstrap.cc:492] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: No bootstrap required, opened a new log
I20260809 08:42:31.166750 10381 ts_tablet_manager.cc:1397] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:42:31.167517 10377 raft_consensus.cc:348] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 } } peers { permanent_uuid: "f4009453748648e8aa37ea67947bcb00" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34803 } } peers { permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 } }
I20260809 08:42:31.167681 10377 raft_consensus.cc:374] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:31.167716 10377 raft_consensus.cc:729] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4009453748648e8aa37ea67947bcb00, State: Initialized, Role: FOLLOWER
I20260809 08:42:31.167891 10377 consensus_queue.cc:260] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 } } peers { permanent_uuid: "f4009453748648e8aa37ea67947bcb00" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34803 } } peers { permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 } }
I20260809 08:42:31.167883 10379 raft_consensus.cc:348] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 } } peers { permanent_uuid: "f4009453748648e8aa37ea67947bcb00" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34803 } } peers { permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 } }
I20260809 08:42:31.168051 10379 raft_consensus.cc:374] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:31.168089 10379 raft_consensus.cc:729] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e27e0401e54d88a1217cbe1ff301e8, State: Initialized, Role: FOLLOWER
I20260809 08:42:31.168120 10389 raft_consensus.cc:482] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:42:31.168190 10389 raft_consensus.cc:504] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 } } peers { permanent_uuid: "f4009453748648e8aa37ea67947bcb00" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34803 } } peers { permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 } }
I20260809 08:42:31.168238 10379 consensus_queue.cc:260] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 } } peers { permanent_uuid: "f4009453748648e8aa37ea67947bcb00" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34803 } } peers { permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 } }
I20260809 08:42:31.168497 10379 ts_tablet_manager.cc:1428] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:42:31.168569 10377 ts_tablet_manager.cc:1428] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:42:31.168973 9939 heartbeater.cc:503] Master 127.31.63.126:33565 was elected leader, sending a full tablet report...
I20260809 08:42:31.169488 10140 heartbeater.cc:503] Master 127.31.63.126:33565 was elected leader, sending a full tablet report...
I20260809 08:42:31.169436 10381 raft_consensus.cc:348] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 } } peers { permanent_uuid: "f4009453748648e8aa37ea67947bcb00" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34803 } } peers { permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 } }
I20260809 08:42:31.169638 10381 raft_consensus.cc:374] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:42:31.169683 10381 raft_consensus.cc:729] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d14dbfb74a142e5a9e5b9552ccabfa0, State: Initialized, Role: FOLLOWER
I20260809 08:42:31.169997 10381 consensus_queue.cc:260] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 } } peers { permanent_uuid: "f4009453748648e8aa37ea67947bcb00" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34803 } } peers { permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 } }
I20260809 08:42:31.170351 10381 ts_tablet_manager.cc:1428] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 08:42:31.170509 10333 heartbeater.cc:503] Master 127.31.63.126:33565 was elected leader, sending a full tablet report...
I20260809 08:42:31.170786 10389 leader_election.cc:288] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3e27e0401e54d88a1217cbe1ff301e8 (127.31.63.66:41417), 0d14dbfb74a142e5a9e5b9552ccabfa0 (127.31.63.67:36907)
I20260809 08:42:31.173657 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.168672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:41417 (local address 127.31.63.65:53569)
0809 08:42:31.168777 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:31.168789 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:31.168833 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:31.170920 (+ 2087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:31.170922 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:31.170924 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:31.170999 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.171001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.171975 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.171976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.172839 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.172842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.173363 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.173364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.173449 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.173453 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:42:31.173509 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 08:42:31.173578 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":1}
I20260809 08:42:31.173856 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.168922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:53569 (local address 127.31.63.66:41417)
0809 08:42:31.170415 (+ 1493us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.170417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.170425 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:31.170446 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.170447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.170457 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.171099 (+ 642us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:31.171134 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.171844 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.172924 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.173289 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.173341 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.173349 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.173748 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.173799 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1470}
I20260809 08:42:31.173910 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.170924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:36907 (local address 127.31.63.65:35341)
0809 08:42:31.171004 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:31.171014 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:42:31.171051 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:42:31.171334 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:42:31.171335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:42:31.171337 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:42:31.171400 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.171402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.172234 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.172235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.172979 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:42:31.172981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.173655 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.173656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:42:31.173749 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.173752 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:42:31.173802 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:42:31.173837 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":1}
I20260809 08:42:31.174062 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.171124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:35341 (local address 127.31.63.67:36907)
0809 08:42:31.171202 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.171204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.171211 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:42:31.171232 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.171233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.171241 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.171290 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:42:31.171459 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.172144 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.173073 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.173530 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.173607 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.173615 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.173979 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.174024 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 08:42:31.174309 10070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c1964a0e5146109ecc3c71c0a1f46b" candidate_uuid: "f4009453748648e8aa37ea67947bcb00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" is_pre_election: true
I20260809 08:42:31.174407 10070 raft_consensus.cc:2454] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4009453748648e8aa37ea67947bcb00 in term 0.
I20260809 08:42:31.174561 10265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c1964a0e5146109ecc3c71c0a1f46b" candidate_uuid: "f4009453748648e8aa37ea67947bcb00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" is_pre_election: true
I20260809 08:42:31.174629 9784 leader_election.cc:302] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e27e0401e54d88a1217cbe1ff301e8, f4009453748648e8aa37ea67947bcb00; no voters:
I20260809 08:42:31.174717 10265 raft_consensus.cc:2454] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4009453748648e8aa37ea67947bcb00 in term 0.
I20260809 08:42:31.174803 10391 raft_consensus.cc:2781] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:42:31.174901 10391 raft_consensus.cc:482] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:42:31.174938 10391 raft_consensus.cc:3037] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:31.175673 10391 raft_consensus.cc:504] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 } } peers { permanent_uuid: "f4009453748648e8aa37ea67947bcb00" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34803 } } peers { permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 } }
I20260809 08:42:31.175814 10391 leader_election.cc:288] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [CANDIDATE]: Term 1 election: Requested vote from peers a3e27e0401e54d88a1217cbe1ff301e8 (127.31.63.66:41417), 0d14dbfb74a142e5a9e5b9552ccabfa0 (127.31.63.67:36907)
I20260809 08:42:31.175959 10070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c1964a0e5146109ecc3c71c0a1f46b" candidate_uuid: "f4009453748648e8aa37ea67947bcb00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3e27e0401e54d88a1217cbe1ff301e8"
I20260809 08:42:31.176017 10265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c1964a0e5146109ecc3c71c0a1f46b" candidate_uuid: "f4009453748648e8aa37ea67947bcb00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0"
I20260809 08:42:31.176034 10070 raft_consensus.cc:3037] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:31.176112 10265 raft_consensus.cc:3037] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:42:31.176749 10070 raft_consensus.cc:2454] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4009453748648e8aa37ea67947bcb00 in term 1.
I20260809 08:42:31.176913 9784 leader_election.cc:302] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e27e0401e54d88a1217cbe1ff301e8, f4009453748648e8aa37ea67947bcb00; no voters:
I20260809 08:42:31.176962 10265 raft_consensus.cc:2454] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4009453748648e8aa37ea67947bcb00 in term 1.
I20260809 08:42:31.177003 10391 raft_consensus.cc:2781] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:42:31.177059 10391 raft_consensus.cc:686] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 1 LEADER]: Becoming Leader. State: Replica: f4009453748648e8aa37ea67947bcb00, State: Running, Role: LEADER
I20260809 08:42:31.177148 10391 consensus_queue.cc:237] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 } } peers { permanent_uuid: "f4009453748648e8aa37ea67947bcb00" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34803 } } peers { permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 } }
I20260809 08:42:31.178112 8566 catalog_manager.cc:5495] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4009453748648e8aa37ea67947bcb00 (127.31.63.65). New cstate: current_term: 1 leader_uuid: "f4009453748648e8aa37ea67947bcb00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4009453748648e8aa37ea67947bcb00" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 } health_report { overall_health: UNKNOWN } } }
I20260809 08:42:31.184387 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.182110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.31.63.65:34803)
0809 08:42:31.182170 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.182172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.182182 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:42:31.182226 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.182227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.182242 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.182300 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:42:31.182452 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.182917 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.183609 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.183799 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.183851 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.184051 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:42:31.184142 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:42:31.184206 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.184277 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.184296 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:42:31.186556 10265 raft_consensus.cc:1261] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0 [term 1 FOLLOWER]: Refusing update from remote peer f4009453748648e8aa37ea67947bcb00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:31.186556 10070 raft_consensus.cc:1261] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8 [term 1 FOLLOWER]: Refusing update from remote peer f4009453748648e8aa37ea67947bcb00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:42:31.186909 10391 consensus_queue.cc:1035] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d14dbfb74a142e5a9e5b9552ccabfa0" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 36907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:31.187048 10389 consensus_queue.cc:1035] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3e27e0401e54d88a1217cbe1ff301e8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:42:31.190354 10399 mvcc.cc:205] Tried to move back new op lower bound from 7316541240056537088 to 7316541240022200320. Current Snapshot: MvccSnapshot[applied={T|T < 7316541240056537088}]
I20260809 08:42:31.191462 10400 mvcc.cc:205] Tried to move back new op lower bound from 7316541240056537088 to 7316541240022200320. Current Snapshot: MvccSnapshot[applied={T|T < 7316541240056537088 or (T in {7316541240056537088})}]
I20260809 08:42:31.193747 8566 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:31.193869 8566 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:42:31.195362 8566 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 021edbb7538f4488976e2bbed5f7f747: Sending DeleteTablet for 3 replicas of tablet a4c1964a0e5146109ecc3c71c0a1f46b
I20260809 08:42:31.195773 9846 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c1964a0e5146109ecc3c71c0a1f46b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:31 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:59672
I20260809 08:42:31.195784 10239 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c1964a0e5146109ecc3c71c0a1f46b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:31 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:33926
I20260809 08:42:31.195852 10044 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c1964a0e5146109ecc3c71c0a1f46b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:42:31 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:53972
I20260809 08:42:31.196156 10411 tablet_replica.cc:330] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: stopping tablet replica
I20260809 08:42:31.196184 10412 tablet_replica.cc:330] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: stopping tablet replica
I20260809 08:42:31.196278 10411 raft_consensus.cc:2229] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:42:31.196424 10412 raft_consensus.cc:2229] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:31.196426 10411 raft_consensus.cc:2258] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:31.196593 10412 raft_consensus.cc:2258] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:31.197180 10412 ts_tablet_manager.cc:1905] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:31.197587 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 8514
I20260809 08:42:31.206209 10412 ts_tablet_manager.cc:1918] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:42:31.206323 10412 log.cc:1196] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/a4c1964a0e5146109ecc3c71c0a1f46b
I20260809 08:42:31.206643 10412 ts_tablet_manager.cc:1939] T a4c1964a0e5146109ecc3c71c0a1f46b P 0d14dbfb74a142e5a9e5b9552ccabfa0: Deleting consensus metadata
I20260809 08:42:31.208206 10414 tablet_replica.cc:330] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: stopping tablet replica
I20260809 08:42:31.208374 10414 raft_consensus.cc:2229] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:42:31.208498 10414 raft_consensus.cc:2258] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:42:31.214198 10411 ts_tablet_manager.cc:1905] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:42:31.214457 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.197169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33062 (local address 127.31.63.124:38357)
0809 08:42:31.197221 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.197223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.197492 (+ 269us) server_negotiation.cc:407] Connection header received
0809 08:42:31.197655 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.197656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.197675 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.197711 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:31.208096 (+ 10385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.208856 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.209730 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.209959 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.210054 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.211435 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:31.211437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:31.211437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:31.211445 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:31.213018 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.213408 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.213410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.213410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.213474 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.213752 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.213753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.213754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.213897 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:31.213980 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.214233 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.214344 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:42:31.215219 10414 ts_tablet_manager.cc:1905] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 08:42:31.216104 10173 connection.cc:535] server connection from 127.0.0.1:33926 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 08:42:31.216368 10411 ts_tablet_manager.cc:1918] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:42:31.216490 10411 log.cc:1196] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/a4c1964a0e5146109ecc3c71c0a1f46b
I20260809 08:42:31.216822 10411 ts_tablet_manager.cc:1939] T a4c1964a0e5146109ecc3c71c0a1f46b P f4009453748648e8aa37ea67947bcb00: Deleting consensus metadata
I20260809 08:42:31.217159 10414 ts_tablet_manager.cc:1918] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:42:31.217244 10414 log.cc:1196] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/a4c1964a0e5146109ecc3c71c0a1f46b
W20260809 08:42:31.217298 9786 connection.cc:441] server connection from 127.0.0.1:59672 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59672 (request call id 1) could send its response
I20260809 08:42:31.217586 10414 ts_tablet_manager.cc:1939] T a4c1964a0e5146109ecc3c71c0a1f46b P a3e27e0401e54d88a1217cbe1ff301e8: Deleting consensus metadata
W20260809 08:42:31.217947 9979 connection.cc:441] server connection from 127.0.0.1:53972 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53972 (request call id 1) could send its response
I20260809 08:42:31.227259 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.219027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.31.63.125:43031)
0809 08:42:31.219067 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.219069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.219232 (+ 163us) server_negotiation.cc:407] Connection header received
0809 08:42:31.219353 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.219354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.219369 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.219428 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:31.220505 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.221309 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.222418 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.222654 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.222741 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.224160 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:31.224162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:31.224162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:31.224171 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:31.225925 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.226327 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.226329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.226329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.226396 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.226615 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.226616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.226616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.226753 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:31.226833 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.227044 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.227140 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:42:31.227274 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:31.219258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33068 (local address 127.31.63.124:38357)
0809 08:42:31.219295 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:42:31.219297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:42:31.219305 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:42:31.219478 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:42:31.219479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:42:31.219512 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:42:31.219565 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:42:31.220232 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.220928 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.222629 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:42:31.222935 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:42:31.222977 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:42:31.224420 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:42:31.224421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:42:31.224422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:42:31.224429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:42:31.226112 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.226472 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.226473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.226474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.226532 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:42:31.226744 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:42:31.226745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:42:31.226745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:42:31.226863 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:42:31.226964 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 08:42:31.227071 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 08:42:31.227152 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:42:32.192135 10425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:32.191788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:46965)
0809 08:42:32.191943 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:32.192090 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260809 08:42:32.192096 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:32.191805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:42331)
0809 08:42:32.191951 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:32.192034 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
W20260809 08:42:32.192634 10333 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:33565 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
I20260809 08:42:32.192803 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:32.192009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:49449)
0809 08:42:32.192671 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:32.192749 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":63,"threads_started":1}
W20260809 08:42:32.192824 10140 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:33565 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
W20260809 08:42:32.193050 9939 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:33565 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
I20260809 08:42:32.696149 10429 raft_consensus.cc:482] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 021edbb7538f4488976e2bbed5f7f747)
I20260809 08:42:32.696345 10429 raft_consensus.cc:504] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } }
I20260809 08:42:32.696681 10429 leader_election.cc:288] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565)
I20260809 08:42:32.697104 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:32.696906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:54954)
0809 08:42:32.697012 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:32.697067 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":63,"threads_started":1}
W20260809 08:42:32.697290 9392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
I20260809 08:42:32.697778 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:32.697671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:54968)
0809 08:42:32.697721 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:32.697749 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 08:42:32.697872 9392 leader_election.cc:334] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565): Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
I20260809 08:42:32.697924 9392 leader_election.cc:302] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: d191b1bebc934a6ba4251a1055cf4bb3; no voters: 021edbb7538f4488976e2bbed5f7f747
I20260809 08:42:32.698012 10429 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:42:33.193539 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:33.193040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:49397)
0809 08:42:33.193427 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:33.193477 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":95,"threads_started":1}
I20260809 08:42:33.193681 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:33.193217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:35959)
0809 08:42:33.193584 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:33.193626 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":56,"threads_started":1}
I20260809 08:42:33.193751 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:33.193425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:46759)
0809 08:42:33.193653 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:33.193700 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":61,"threads_started":1}
I20260809 08:42:34.194463 10438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:34.194093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:52327)
0809 08:42:34.194360 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:34.194403 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":70,"threads_started":1}
I20260809 08:42:34.194497 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:34.194246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:35435)
0809 08:42:34.194423 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:34.194453 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":52,"threads_started":1}
I20260809 08:42:34.194564 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:34.194097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:50537)
0809 08:42:34.194467 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:34.194510 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":110,"threads_started":1}
I20260809 08:42:34.490870 10441 raft_consensus.cc:482] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 021edbb7538f4488976e2bbed5f7f747)
I20260809 08:42:34.490969 10441 raft_consensus.cc:504] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } }
I20260809 08:42:34.491178 10441 leader_election.cc:288] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565)
I20260809 08:42:34.491557 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:34.491352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:55218)
0809 08:42:34.491480 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:34.491523 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260809 08:42:34.491971 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:34.491893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:55222)
0809 08:42:34.491925 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:34.491951 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 08:42:34.492058 9392 leader_election.cc:334] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565): Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
I20260809 08:42:34.492136 9392 leader_election.cc:302] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: d191b1bebc934a6ba4251a1055cf4bb3; no voters: 021edbb7538f4488976e2bbed5f7f747
I20260809 08:42:34.492218 10441 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:42:35.195310 10446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:35.195048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:48661)
0809 08:42:35.195172 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:35.195215 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260809 08:42:35.195323 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:35.195013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:54019)
0809 08:42:35.195247 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:35.195285 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":70,"threads_started":1}
I20260809 08:42:35.195353 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:35.195141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:44695)
0809 08:42:35.195262 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:35.195301 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":50,"threads_started":1}
I20260809 08:42:36.196058 10449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:36.195765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:40787)
0809 08:42:36.195940 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:36.195998 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":69,"threads_started":1}
I20260809 08:42:36.196051 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:36.195725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:50739)
0809 08:42:36.195939 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:36.195989 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":75,"threads_started":1}
I20260809 08:42:36.196147 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:36.195907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:38251)
0809 08:42:36.196072 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:36.196104 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":49,"threads_started":1}
I20260809 08:42:37.196774 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:37.196493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:38801)
0809 08:42:37.196649 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:37.196712 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260809 08:42:37.196830 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:37.196623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:43161)
0809 08:42:37.196728 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:37.196769 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":57,"threads_started":1}
I20260809 08:42:37.197064 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:37.196748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:44999)
0809 08:42:37.196879 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:37.197017 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260809 08:42:37.862352 10456 raft_consensus.cc:482] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 021edbb7538f4488976e2bbed5f7f747)
I20260809 08:42:37.862473 10456 raft_consensus.cc:504] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } }
I20260809 08:42:37.862643 10456 leader_election.cc:288] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565)
I20260809 08:42:37.863031 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:37.862837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:45094)
0809 08:42:37.862942 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:37.862990 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":59,"threads_started":1}
W20260809 08:42:37.863122 9392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260809 08:42:37.863508 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:37.863393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:45110)
0809 08:42:37.863449 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:37.863476 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 08:42:37.863585 9392 leader_election.cc:334] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565): Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
I20260809 08:42:37.863651 9392 leader_election.cc:302] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: d191b1bebc934a6ba4251a1055cf4bb3; no voters: 021edbb7538f4488976e2bbed5f7f747
I20260809 08:42:37.863735 10456 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:42:38.197580 10460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:38.197305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:57983)
0809 08:42:38.197453 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:38.197488 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260809 08:42:38.197593 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:38.197305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:57745)
0809 08:42:38.197449 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:38.197499 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260809 08:42:38.197757 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:38.197494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:36275)
0809 08:42:38.197675 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:38.197711 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":118,"threads_started":1}
I20260809 08:42:39.198325 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:39.198051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:56227)
0809 08:42:39.198213 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:39.198263 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260809 08:42:39.198370 10465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:39.198157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:35579)
0809 08:42:39.198274 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:39.198316 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260809 08:42:39.198407 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:39.198190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:49723)
0809 08:42:39.198328 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:39.198360 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260809 08:42:40.199097 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:40.198824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:44901)
0809 08:42:40.198989 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:40.199037 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260809 08:42:40.199098 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:40.198824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:42407)
0809 08:42:40.198990 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:40.199035 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":79,"threads_started":1}
I20260809 08:42:40.199183 10474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:40.198881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:54681)
0809 08:42:40.199105 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:40.199140 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":81,"threads_started":1}
I20260809 08:42:41.199832 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:41.199578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:46305)
0809 08:42:41.199737 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:41.199779 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260809 08:42:41.199832 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:41.199578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:50707)
0809 08:42:41.199742 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:41.199779 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":71,"threads_started":1}
I20260809 08:42:41.199910 10477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:41.199690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:51465)
0809 08:42:41.199818 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:41.199857 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
I20260809 08:42:42.200628 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:42.200334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:54779)
0809 08:42:42.200509 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:42.200565 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
I20260809 08:42:42.200783 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:42.200521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:33401)
0809 08:42:42.200689 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:42.200731 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":48,"threads_started":1}
I20260809 08:42:42.200853 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:42.200333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:55765)
0809 08:42:42.200773 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:42.200814 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":77,"threads_started":1}
I20260809 08:42:42.341713 10484 raft_consensus.cc:482] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 021edbb7538f4488976e2bbed5f7f747)
I20260809 08:42:42.341816 10484 raft_consensus.cc:504] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } }
I20260809 08:42:42.341946 10484 leader_election.cc:288] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565)
I20260809 08:42:42.342290 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:42.342099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:45418)
0809 08:42:42.342207 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:42.342251 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":61,"threads_started":1}
I20260809 08:42:42.342717 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:42.342620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:45430)
0809 08:42:42.342668 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:42.342698 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 08:42:42.342799 9392 leader_election.cc:334] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565): Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
I20260809 08:42:42.342844 9392 leader_election.cc:302] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: d191b1bebc934a6ba4251a1055cf4bb3; no voters: 021edbb7538f4488976e2bbed5f7f747
I20260809 08:42:42.343037 10484 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:42:43.201646 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:43.201216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:39575)
0809 08:42:43.201552 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:43.201596 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":71,"threads_started":1}
I20260809 08:42:43.201645 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:43.201214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:52679)
0809 08:42:43.201473 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:43.201520 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":69,"threads_started":1}
I20260809 08:42:43.201764 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:43.201477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:37415)
0809 08:42:43.201666 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:43.201711 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":118,"threads_started":1}
I20260809 08:42:44.202445 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:44.202164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:51033)
0809 08:42:44.202356 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:44.202391 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":68,"threads_started":1}
I20260809 08:42:44.202445 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:44.202165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:49659)
0809 08:42:44.202343 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:44.202390 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260809 08:42:44.202802 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:44.202438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:45973)
0809 08:42:44.202580 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:44.202752 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":59,"threads_started":1}
I20260809 08:42:45.203507 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:45.203205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:40841)
0809 08:42:45.203386 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:45.203431 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":48,"threads_started":1}
I20260809 08:42:45.203532 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:45.203091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:33607)
0809 08:42:45.203429 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:45.203480 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":73,"threads_started":1}
I20260809 08:42:45.203711 10499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:45.203316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:49703)
0809 08:42:45.203561 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:45.203597 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":65,"threads_started":1}
I20260809 08:42:46.204272 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:46.204015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:47861)
0809 08:42:46.204169 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:46.204212 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
I20260809 08:42:46.204360 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:46.204115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:42709)
0809 08:42:46.204250 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:46.204298 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":51,"threads_started":1}
I20260809 08:42:46.204615 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:46.204296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:46731)
0809 08:42:46.204531 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:46.204573 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":61,"threads_started":1}
I20260809 08:42:47.205163 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:47.204886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:52981)
0809 08:42:47.205034 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:47.205081 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260809 08:42:47.205335 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:47.205012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:50295)
0809 08:42:47.205251 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:47.205292 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":121,"threads_started":1}
I20260809 08:42:47.205327 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:47.205101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:48135)
0809 08:42:47.205223 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:47.205271 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":52,"threads_started":1}
I20260809 08:42:48.206276 10512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:48.205948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:44673)
0809 08:42:48.206164 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:48.206211 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":65,"threads_started":1}
I20260809 08:42:48.206354 10513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:48.206099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:51857)
0809 08:42:48.206265 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:48.206306 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
I20260809 08:42:48.206354 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:48.205810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:52509)
0809 08:42:48.206268 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:48.206306 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":391,"threads_started":1}
I20260809 08:42:49.141009 10516 raft_consensus.cc:482] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 021edbb7538f4488976e2bbed5f7f747)
I20260809 08:42:49.141165 10516 raft_consensus.cc:504] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } }
I20260809 08:42:49.141501 10516 leader_election.cc:288] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565)
I20260809 08:42:49.142146 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:49.141783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:49476)
0809 08:42:49.141987 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:49.142060 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":80,"threads_started":1}
W20260809 08:42:49.142272 9392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260809 08:42:49.142773 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:49.142679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:49484)
0809 08:42:49.142726 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:49.142754 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 08:42:49.142866 9392 leader_election.cc:334] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565): Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
I20260809 08:42:49.142980 9392 leader_election.cc:302] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: d191b1bebc934a6ba4251a1055cf4bb3; no voters: 021edbb7538f4488976e2bbed5f7f747
I20260809 08:42:49.143154 10516 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:42:49.207975 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:49.207435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:45207)
0809 08:42:49.207759 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:49.207848 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":111,"threads_started":1}
I20260809 08:42:49.208069 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:49.206964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:43245)
0809 08:42:49.207907 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:49.207972 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":916,"thread_start_us":170,"threads_started":1}
I20260809 08:42:49.208297 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:49.207330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:34649)
0809 08:42:49.208108 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:49.208198 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":129,"threads_started":1}
I20260809 08:42:50.209400 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:50.209064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:47799)
0809 08:42:50.209235 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:50.209279 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":69,"threads_started":1}
I20260809 08:42:50.209400 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:50.208950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:46745)
0809 08:42:50.209215 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:50.209277 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":136,"threads_started":1}
I20260809 08:42:50.209421 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:50.209007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:39051)
0809 08:42:50.209192 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:50.209257 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260809 08:42:51.210220 10532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:51.209938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:52397)
0809 08:42:51.210108 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:51.210157 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260809 08:42:51.210306 10536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:51.210044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:47017)
0809 08:42:51.210228 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:51.210263 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":50,"threads_started":1}
I20260809 08:42:51.210256 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:51.209945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:59897)
0809 08:42:51.210156 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:51.210201 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":71,"threads_started":1}
I20260809 08:42:51.838755 10537 raft_consensus.cc:482] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 021edbb7538f4488976e2bbed5f7f747)
I20260809 08:42:51.838865 10537 raft_consensus.cc:504] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "021edbb7538f4488976e2bbed5f7f747" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33565 } } peers { permanent_uuid: "d191b1bebc934a6ba4251a1055cf4bb3" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 43031 } attrs { promote: false } } peers { permanent_uuid: "3cb1546ac394456396e1d828395c60a9" member_type: NON_VOTER last_known_addr { host: "127.31.63.124" port: 38357 } attrs { promote: true } }
I20260809 08:42:51.838997 10537 leader_election.cc:288] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565)
I20260809 08:42:51.839493 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:51.839223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:49602)
0809 08:42:51.839372 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:51.839420 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260809 08:42:51.840029 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:51.839904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.0.0.1:49604)
0809 08:42:51.839973 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:51.840006 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 08:42:51.840114 9392 leader_election.cc:334] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 021edbb7538f4488976e2bbed5f7f747 (127.31.63.126:33565): Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
I20260809 08:42:51.840189 9392 leader_election.cc:302] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: d191b1bebc934a6ba4251a1055cf4bb3; no voters: 021edbb7538f4488976e2bbed5f7f747
I20260809 08:42:51.840307 10537 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d191b1bebc934a6ba4251a1055cf4bb3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:42:52.210907 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:52.210685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:44365)
0809 08:42:52.210809 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:52.210851 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":50,"threads_started":1}
I20260809 08:42:52.210912 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:52.210660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:48753)
0809 08:42:52.210809 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:52.210856 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260809 08:42:52.211130 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:52.210876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:43875)
0809 08:42:52.211034 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:52.211076 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260809 08:42:53.211642 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:53.211388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:56575)
0809 08:42:53.211527 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:53.211565 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260809 08:42:53.211736 10546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:53.211535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:33267)
0809 08:42:53.211652 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:53.211690 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":45,"threads_started":1}
I20260809 08:42:53.211736 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:53.211436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:54143)
0809 08:42:53.211653 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:53.211693 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":82,"threads_started":1}
I20260809 08:42:54.212399 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:54.212124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:57519)
0809 08:42:54.212285 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:54.212325 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260809 08:42:54.212399 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:54.212117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:49681)
0809 08:42:54.212285 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:54.212326 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":76,"threads_started":1}
I20260809 08:42:54.212563 10553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:54.212257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:37263)
0809 08:42:54.212402 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:54.212509 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":56,"threads_started":1}
I20260809 08:42:55.213032 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:55.212851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:36667)
0809 08:42:55.212966 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:55.212993 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":51,"threads_started":1}
I20260809 08:42:55.213032 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:55.212782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:35337)
0809 08:42:55.212931 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:55.212974 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260809 08:42:55.213166 10557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:55.212943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:37569)
0809 08:42:55.213052 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:55.213091 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
I20260809 08:42:56.213732 10559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:56.213432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:52059)
0809 08:42:56.213590 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:56.213659 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260809 08:42:56.213929 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:56.213661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:51859)
0809 08:42:56.213832 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:56.213873 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":58,"threads_started":1}
I20260809 08:42:56.213976 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:56.213432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:49187)
0809 08:42:56.213899 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:56.213932 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":77,"threads_started":1}
I20260809 08:42:57.214704 10563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:57.214386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:48067)
0809 08:42:57.214617 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:57.214656 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":61,"threads_started":1}
I20260809 08:42:57.214543 10561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:57.214287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:60533)
0809 08:42:57.214447 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:57.214496 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260809 08:42:57.214809 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:57.214429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:51249)
0809 08:42:57.214645 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:57.214675 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":50,"threads_started":1}
I20260809 08:42:58.215467 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:58.215203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:52649)
0809 08:42:58.215363 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:58.215410 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260809 08:42:58.215520 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:58.215307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:51419)
0809 08:42:58.215427 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:58.215468 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260809 08:42:58.215554 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:58.215203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:42045)
0809 08:42:58.215479 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:58.215509 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":78,"threads_started":1}
I20260809 08:42:59.216302 10572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:59.215991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:38543)
0809 08:42:59.216224 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:59.216258 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":76,"threads_started":1}
I20260809 08:42:59.216264 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:59.215991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:50269)
0809 08:42:59.216142 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:59.216194 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260809 08:42:59.216583 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:42:59.216029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:54709)
0809 08:42:59.216470 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 08:42:59.216532 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":77,"threads_started":1}
I20260809 08:43:00.217026 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:00.216758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:55219)
0809 08:43:00.216923 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:00.216967 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260809 08:43:00.217064 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:00.216851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:57825)
0809 08:43:00.216965 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:00.217010 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260809 08:43:00.217338 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:00.217061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:55567)
0809 08:43:00.217256 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:00.217294 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":54,"threads_started":1}
I20260809 08:43:01.217924 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:01.217605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.66:55265)
0809 08:43:01.217823 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:01.217869 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":64,"threads_started":1}
I20260809 08:43:01.217924 10580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:01.217707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.67:40531)
0809 08:43:01.217825 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:01.217867 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":53,"threads_started":1}
I20260809 08:43:01.218036 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:01.217757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33565 (local address 127.31.63.65:58217)
0809 08:43:01.217884 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:01.217912 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:33565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.31.63.126:33565,127.31.63.125:43031,127.31.63.124:38357, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.31.63.126:33565,127.31.63.125:43031,127.31.63.124:38357, num_attempts: 1)
I20260809 08:43:01.430539 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 9632
I20260809 08:43:01.446565 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 9948
I20260809 08:43:01.455050 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 10147
I20260809 08:43:01.464517 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 9220
I20260809 08:43:01.477124 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 9499
2026-08-09T08:43:01Z chronyd exiting
I20260809 08:43:01.502074 31997 test_util.cc:175] -----------------------------------------------
I20260809 08:43:01.502136 31997 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786264918773806-31997-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (33523 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:43:01 dist-test-slave-gkw7 krb5kdc[10586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:43:01 dist-test-slave-gkw7 krb5kdc[10586](info): set up 2 sockets
Aug 09 08:43:01 dist-test-slave-gkw7 krb5kdc[10586](info): commencing operation
krb5kdc: starting...
W20260809 08:43:02.761391 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:43:02 dist-test-slave-gkw7 krb5kdc[10586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:43:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:43:02Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.126.keytab.
Entry for principal oryx/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.126.keytab.
I20260809 08:43:02.894871 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:42639
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.126.keytab
--principal=oryx/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:42639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:02.980568 10602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:02.980818 10602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:02.980854 10602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:02.982074 10602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:43:02.982149 10602 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 08:43:02.982170 10602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:02.982192 10602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:02.982225 10602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:43:02.982247 10602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:43:02.983771 10602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:42639
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.126.keytab
--principal=oryx/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:42639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:02.984061 10602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:02.984294 10602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:43:02.989756 10602 server_base.cc:1034] running on GCE node
W20260809 08:43:02.989727 10609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:02.989719 10617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:02.989727 10607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:02.990342 10602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:02.990657 10602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:02.991808 10602 hybrid_clock.cc:648] HybridClock initialized: now 1786264982991798 us; error 31 us; skew 500 ppm
Aug 09 08:43:02 dist-test-slave-gkw7 krb5kdc[10586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264982, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:02.994392 10602 init.cc:377] Logged in from keytab as oryx/127.31.63.126@KRBTEST.COM (short username oryx)
I20260809 08:43:02.994832 10602 webserver.cc:460] Webserver started at http://127.31.63.126:35825/ using document root <none> and password file <none>
I20260809 08:43:02.995056 10602 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:02.995101 10602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:02.995260 10602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:02.996364 10602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e"
format_stamp: "Formatted at 2026-08-09 08:43:02 on dist-test-slave-gkw7"
server_key: "ac1cdc6534023ab5b12c089358048979"
server_key_iv: "92979fcc6f6b919afc0b9f0e8e718fd0"
server_key_version: "encryptionkey@0"
I20260809 08:43:02.996743 10602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e"
format_stamp: "Formatted at 2026-08-09 08:43:02 on dist-test-slave-gkw7"
server_key: "ac1cdc6534023ab5b12c089358048979"
server_key_iv: "92979fcc6f6b919afc0b9f0e8e718fd0"
server_key_version: "encryptionkey@0"
I20260809 08:43:02.998236 10602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:43:02.999004 10626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:02.999202 10602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:43:02.999302 10602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e"
format_stamp: "Formatted at 2026-08-09 08:43:02 on dist-test-slave-gkw7"
server_key: "ac1cdc6534023ab5b12c089358048979"
server_key_iv: "92979fcc6f6b919afc0b9f0e8e718fd0"
server_key_version: "encryptionkey@0"
I20260809 08:43:02.999379 10602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:03.018244 10602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:03.020205 10602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:03.020355 10602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:03.025599 10602 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:42639
I20260809 08:43:03.025700 10714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:42639 every 8 connection(s)
I20260809 08:43:03.026067 10602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:43:03.026813 10715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:03.029223 10715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e: Bootstrap starting.
I20260809 08:43:03.029954 10715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:03.030265 10715 log.cc:824] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:03.030982 10715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e: No bootstrap required, opened a new log
I20260809 08:43:03.031200 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 10602
I20260809 08:43:03.031332 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:43:03.031549 31997 external_mini_cluster.cc:1428] Setting key 8636f64f1e28109f9b0622b9722ea353
I20260809 08:43:03.033501 10715 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42639 } }
I20260809 08:43:03.033883 10715 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:03.033958 10715 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2ba1c415c2841f481b7e7feb9ac4d9e, State: Initialized, Role: FOLLOWER
I20260809 08:43:03.034098 10715 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42639 } }
I20260809 08:43:03.034198 10715 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:43:03.034236 10715 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:43:03.034293 10715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:03.034907 10715 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42639 } }
I20260809 08:43:03.035029 10715 leader_election.cc:302] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2ba1c415c2841f481b7e7feb9ac4d9e; no voters:
I20260809 08:43:03.035216 10715 leader_election.cc:288] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:43:03.035248 10720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:03.035392 10720 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [term 1 LEADER]: Becoming Leader. State: Replica: c2ba1c415c2841f481b7e7feb9ac4d9e, State: Running, Role: LEADER
I20260809 08:43:03.035531 10715 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:43:03.035531 10720 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42639 } }
I20260809 08:43:03.035887 10720 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2ba1c415c2841f481b7e7feb9ac4d9e. Latest consensus state: current_term: 1 leader_uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42639 } } }
I20260809 08:43:03.035902 10721 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2ba1c415c2841f481b7e7feb9ac4d9e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42639 } } }
I20260809 08:43:03.035979 10720 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:03.036003 10721 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:03.036175 10725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:43:03.036757 10725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:43:03.038638 10725 catalog_manager.cc:1347] Generated new cluster ID: 018668740be94fb7adcf99135f8232fd
I20260809 08:43:03.038697 10725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:43:03.043648 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.032113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43884 (local address 127.31.63.126:42639)
0809 08:43:03.032301 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.032309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.032335 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:43:03.033485 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.033494 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.033608 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.033733 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:03.034542 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.035757 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.036859 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.037584 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.037691 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.040217 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:03.040221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:03.040222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:03.040234 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:03.042026 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.042410 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.042414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.042414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.042522 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.042740 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.042741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.042741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.042984 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:03.043170 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.043240 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.043498 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260809 08:43:03.051167 10725 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:43:03.051710 10725 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:43:03.061144 10725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e: Generated new TSK 0
I20260809 08:43:03.061352 10725 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab.
Entry for principal oryx/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab.
I20260809 08:43:03.092666 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42639
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab
--principal=oryx/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:03.177634 10748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:03.177933 10748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:03.177961 10748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:03.179132 10748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:03.179174 10748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:03.179195 10748 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:43:03.179250 10748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:43:03.180840 10748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab
--principal=oryx/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.63.126:42639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:03.181095 10748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:03.181339 10748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:03.186496 10755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:03.186479 10759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:03.186502 10756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:03.186741 10748 server_base.cc:1034] running on GCE node
I20260809 08:43:03.187002 10748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:03.187252 10748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:03.188422 10748 hybrid_clock.cc:648] HybridClock initialized: now 1786264983188417 us; error 38 us; skew 500 ppm
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264983, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:03.191112 10748 init.cc:377] Logged in from keytab as oryx/127.31.63.65@KRBTEST.COM (short username oryx)
I20260809 08:43:03.191563 10748 webserver.cc:460] Webserver started at http://127.31.63.65:44711/ using document root <none> and password file <none>
I20260809 08:43:03.191754 10748 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:03.191803 10748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:03.191895 10748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:03.192884 10748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "f7d927b63ca04493b9363616bbc03660"
format_stamp: "Formatted at 2026-08-09 08:43:03 on dist-test-slave-gkw7"
server_key: "2f794a17323f17f5ac38a78d18ada497"
server_key_iv: "c844d032384a3e2c8c18a4f4deff0267"
server_key_version: "encryptionkey@0"
I20260809 08:43:03.193305 10748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "f7d927b63ca04493b9363616bbc03660"
format_stamp: "Formatted at 2026-08-09 08:43:03 on dist-test-slave-gkw7"
server_key: "2f794a17323f17f5ac38a78d18ada497"
server_key_iv: "c844d032384a3e2c8c18a4f4deff0267"
server_key_version: "encryptionkey@0"
I20260809 08:43:03.194697 10748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 08:43:03.195402 10767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:03.195585 10748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:43:03.195658 10748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "f7d927b63ca04493b9363616bbc03660"
format_stamp: "Formatted at 2026-08-09 08:43:03 on dist-test-slave-gkw7"
server_key: "2f794a17323f17f5ac38a78d18ada497"
server_key_iv: "c844d032384a3e2c8c18a4f4deff0267"
server_key_version: "encryptionkey@0"
I20260809 08:43:03.195711 10748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:03.211687 10748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:03.214126 10748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:03.214243 10748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:03.214811 10748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:03.214865 10748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:03.214890 10748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:03.214922 10748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264983, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:43:03.222810 10748 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:35265
I20260809 08:43:03.223167 10748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:43:03.223667 10933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:35265 every 8 connection(s)
I20260809 08:43:03.225449 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.215256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:42661 (local address 127.31.63.126:42639)
0809 08:43:03.215306 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.215308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.215521 (+ 213us) server_negotiation.cc:407] Connection header received
0809 08:43:03.216037 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.216039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.216058 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.216140 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:03.217029 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.217912 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.219020 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.219376 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.219461 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.222301 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:03.222303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:03.222303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:03.222314 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:03.224035 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.224427 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.224429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.224429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.224490 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.224764 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.224765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.224765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.224890 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:03.224968 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.225252 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.225356 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:43:03.225628 10781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.215227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42639 (local address 127.31.63.65:42661)
0809 08:43:03.215398 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.215437 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.215927 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.216214 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.216218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.216619 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:03.216910 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.216917 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.218013 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.218015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.218924 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.218926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.219523 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.219524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.219699 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.220293 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:03.220305 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:03.222192 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:03.224114 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.224121 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.224123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.224347 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.224566 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.224567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.224568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.224685 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.224960 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:03.224968 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:03.225109 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.225217 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.225378 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260809 08:43:03.226267 10934 heartbeater.cc:347] Connected to a master server at 127.31.63.126:42639
I20260809 08:43:03.226372 10934 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:03.226548 10934 heartbeater.cc:511] Master 127.31.63.126:42639 requested a full tablet report, sending...
I20260809 08:43:03.227149 10648 ts_manager.cc:194] Registered new tserver with Master: f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265)
I20260809 08:43:03.228113 10648 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:42661
I20260809 08:43:03.228627 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 10748
I20260809 08:43:03.228749 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:43:03.228965 31997 external_mini_cluster.cc:1428] Setting key 0553603d18153ddf86128da732878ebd
I20260809 08:43:03.233490 10942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.227340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42639 (local address 127.31.63.65:57149)
0809 08:43:03.227474 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.227484 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.227541 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.227660 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.227661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.227853 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:03.227928 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.227929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.228793 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.228794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.229468 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.229470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.229946 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.229947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.230023 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.230443 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:03.230447 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:03.230728 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:03.232473 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.232474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.232474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.232645 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.232861 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.232862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.232862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.232916 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.233202 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:03.233203 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:03.233251 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.233341 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.233420 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260809 08:43:03.233640 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.227385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:57149 (local address 127.31.63.126:42639)
0809 08:43:03.227426 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.227427 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.227528 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:43:03.227595 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.227596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.227604 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.227647 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:03.227992 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.228664 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.229589 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.229854 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.229944 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.230821 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:03.230823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:03.230823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:03.230830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:03.232384 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.232726 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.232727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.232727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.232793 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.232987 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.232988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.232988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.233131 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:03.233210 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.233392 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.233484 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
WARNING: no policy specified for oryx/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab.
Entry for principal oryx/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab.
I20260809 08:43:03.256397 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42639
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab
--principal=oryx/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:03.345170 10946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:03.345372 10946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:03.345396 10946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:03.346494 10946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:03.346529 10946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:03.346549 10946 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:43:03.346565 10946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:43:03.348030 10946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab
--principal=oryx/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.63.126:42639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:03.348223 10946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:03.348397 10946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:03.353436 10954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:03.353607 10946 server_base.cc:1034] running on GCE node
W20260809 08:43:03.353449 10955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:03.353498 10958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:03.353963 10946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:03.354166 10946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:03.355312 10946 hybrid_clock.cc:648] HybridClock initialized: now 1786264983355305 us; error 39 us; skew 500 ppm
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264983, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:03.357800 10946 init.cc:377] Logged in from keytab as oryx/127.31.63.66@KRBTEST.COM (short username oryx)
I20260809 08:43:03.358276 10946 webserver.cc:460] Webserver started at http://127.31.63.66:40549/ using document root <none> and password file <none>
I20260809 08:43:03.358522 10946 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:03.358592 10946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:03.358736 10946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:03.359791 10946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "b29a7e8c2c414b968de49a8d00114007"
format_stamp: "Formatted at 2026-08-09 08:43:03 on dist-test-slave-gkw7"
server_key: "8ed85563402828e2973d7a3c321dd947"
server_key_iv: "543f8ca3e3daf1aa7fcc24cb3a096362"
server_key_version: "encryptionkey@0"
I20260809 08:43:03.360163 10946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "b29a7e8c2c414b968de49a8d00114007"
format_stamp: "Formatted at 2026-08-09 08:43:03 on dist-test-slave-gkw7"
server_key: "8ed85563402828e2973d7a3c321dd947"
server_key_iv: "543f8ca3e3daf1aa7fcc24cb3a096362"
server_key_version: "encryptionkey@0"
I20260809 08:43:03.361657 10946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:43:03.362430 10967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:03.362613 10946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:43:03.362741 10946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "b29a7e8c2c414b968de49a8d00114007"
format_stamp: "Formatted at 2026-08-09 08:43:03 on dist-test-slave-gkw7"
server_key: "8ed85563402828e2973d7a3c321dd947"
server_key_iv: "543f8ca3e3daf1aa7fcc24cb3a096362"
server_key_version: "encryptionkey@0"
I20260809 08:43:03.362831 10946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:03.375151 10946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:03.376899 10946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:03.377059 10946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:03.377738 10946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:03.377804 10946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:03.377838 10946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:03.377895 10946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264983, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:43:03.386418 10946 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:45429
I20260809 08:43:03.386782 10946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:43:03.386782 11119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:45429 every 8 connection(s)
I20260809 08:43:03.388825 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.378167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:55673 (local address 127.31.63.126:42639)
0809 08:43:03.378244 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.378247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.378517 (+ 270us) server_negotiation.cc:407] Connection header received
0809 08:43:03.379286 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.379288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.379301 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.379348 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:03.380295 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.380819 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.382579 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.382894 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.382978 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.385613 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:03.385615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:03.385615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:03.385623 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:03.387365 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.387819 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.387821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.387821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.387884 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.388146 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.388148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.388148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.388281 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:03.388357 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.388646 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.388731 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:43:03.389026 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.378158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42639 (local address 127.31.63.66:55673)
0809 08:43:03.378354 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.378407 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.379171 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.379425 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.379429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.379828 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:03.380168 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.380175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.380934 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.380935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.382455 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.382459 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.383002 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.383003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.383177 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.383782 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:03.383793 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:03.385476 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:03.387461 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.387468 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.387470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.387719 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.387963 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.387964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.387964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.388076 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.388360 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:03.388369 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:03.388511 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.388619 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.388795 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
I20260809 08:43:03.389595 11120 heartbeater.cc:347] Connected to a master server at 127.31.63.126:42639
I20260809 08:43:03.389685 11120 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:03.389883 11120 heartbeater.cc:511] Master 127.31.63.126:42639 requested a full tablet report, sending...
I20260809 08:43:03.390331 10648 ts_manager.cc:194] Registered new tserver with Master: b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429)
I20260809 08:43:03.390750 10648 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:55673
I20260809 08:43:03.392082 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 10946
I20260809 08:43:03.392197 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:43:03.392413 31997 external_mini_cluster.cc:1428] Setting key a4f27f496a0202c8bd1750161837f36d
I20260809 08:43:03.396900 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.390791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:36503 (local address 127.31.63.126:42639)
0809 08:43:03.390835 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.390837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.390916 (+ 79us) server_negotiation.cc:407] Connection header received
0809 08:43:03.390982 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.390983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.390993 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.391056 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:03.391443 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.391921 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.392833 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.393104 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.393168 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.394241 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:03.394242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:03.394242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:03.394248 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:03.395700 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.396050 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.396051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.396051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.396110 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.396338 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.396339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.396339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.396488 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:03.396567 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.396734 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.396810 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:43:03.396916 11127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.390699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42639 (local address 127.31.63.66:36503)
0809 08:43:03.390854 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.390867 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.390929 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.391059 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.391060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.391279 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:03.391362 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.391364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.392024 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.392025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.392761 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.392764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.393192 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.393193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.393277 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.393740 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:03.393746 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:03.394155 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:03.395782 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.395783 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.395784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.395962 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.396200 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.396201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.396201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.396266 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.396560 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:03.396561 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:03.396609 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.396706 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.396811 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for oryx/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab.
Entry for principal oryx/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab.
I20260809 08:43:03.420105 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42639
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab
--principal=oryx/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:03.503988 11133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:03.504194 11133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:03.504217 11133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:03.505438 11133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:03.505483 11133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:03.505503 11133 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:43:03.505573 11133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:43:03.507073 11133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab
--principal=oryx/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.63.126:42639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:03.507326 11133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:03.507567 11133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:03.512844 11144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:03.512876 11145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:03.512876 11147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:03.512894 11133 server_base.cc:1034] running on GCE node
I20260809 08:43:03.513247 11133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:03.513459 11133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:03.514616 11133 hybrid_clock.cc:648] HybridClock initialized: now 1786264983514597 us; error 31 us; skew 500 ppm
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264983, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:03.517139 11133 init.cc:377] Logged in from keytab as oryx/127.31.63.67@KRBTEST.COM (short username oryx)
I20260809 08:43:03.517678 11133 webserver.cc:460] Webserver started at http://127.31.63.67:34451/ using document root <none> and password file <none>
I20260809 08:43:03.517864 11133 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:03.517904 11133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:03.517994 11133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:03.518987 11133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "41f43232019c4e9e9ff3900c642028a3"
format_stamp: "Formatted at 2026-08-09 08:43:03 on dist-test-slave-gkw7"
server_key: "d640a2b40e9eec68a22b732ff0974c60"
server_key_iv: "aafce9b2b5ed43cd28504dc3481fcb75"
server_key_version: "encryptionkey@0"
I20260809 08:43:03.519376 11133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "41f43232019c4e9e9ff3900c642028a3"
format_stamp: "Formatted at 2026-08-09 08:43:03 on dist-test-slave-gkw7"
server_key: "d640a2b40e9eec68a22b732ff0974c60"
server_key_iv: "aafce9b2b5ed43cd28504dc3481fcb75"
server_key_version: "encryptionkey@0"
I20260809 08:43:03.520624 11133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:43:03.521281 11159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:03.521514 11133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:03.521642 11133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "41f43232019c4e9e9ff3900c642028a3"
format_stamp: "Formatted at 2026-08-09 08:43:03 on dist-test-slave-gkw7"
server_key: "d640a2b40e9eec68a22b732ff0974c60"
server_key_iv: "aafce9b2b5ed43cd28504dc3481fcb75"
server_key_version: "encryptionkey@0"
I20260809 08:43:03.521734 11133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:03.530853 11133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:03.532641 11133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:03.532814 11133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:03.533313 11133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:03.533380 11133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:03.533423 11133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:03.533447 11133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264983, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:43:03.541117 11133 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:37063
I20260809 08:43:03.541203 11333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:37063 every 8 connection(s)
I20260809 08:43:03.541584 11133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:43:03.543915 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.533818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:33693 (local address 127.31.63.126:42639)
0809 08:43:03.533891 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.533894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.534071 (+ 177us) server_negotiation.cc:407] Connection header received
0809 08:43:03.534531 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.534533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.534549 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.534634 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:03.535556 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.536453 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.537798 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.538013 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.538092 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.540799 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:03.540801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:03.540802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:03.540811 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:03.542448 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.542882 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.542884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.542885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.542952 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.543207 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.543208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.543208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.543337 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:03.543411 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.543717 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.543822 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:43:03.544052 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.533789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42639 (local address 127.31.63.67:33693)
0809 08:43:03.533935 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.533975 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.534440 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.534684 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.534687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.535103 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:03.535445 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.535452 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.536577 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.536578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.537707 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.537710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.538119 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.538120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.538247 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.538739 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:03.538754 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:03.540677 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:03.542549 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.542557 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.542559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.542782 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.543021 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.543022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.543023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.543135 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.543414 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:03.543422 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:03.543575 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.543677 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.543832 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260809 08:43:03.544549 11334 heartbeater.cc:347] Connected to a master server at 127.31.63.126:42639
I20260809 08:43:03.544652 11334 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:03.544833 11334 heartbeater.cc:511] Master 127.31.63.126:42639 requested a full tablet report, sending...
I20260809 08:43:03.545156 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 11133
I20260809 08:43:03.545249 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:43:03.545491 31997 external_mini_cluster.cc:1428] Setting key fc6a889e24b4c64288015905dabd664a
I20260809 08:43:03.545924 10648 ts_manager.cc:194] Registered new tserver with Master: 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063)
I20260809 08:43:03.546293 10648 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:33693
I20260809 08:43:03.546873 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:43:03.551429 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.545736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:35127 (local address 127.31.63.126:42639)
0809 08:43:03.545780 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.545782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.545859 (+ 77us) server_negotiation.cc:407] Connection header received
0809 08:43:03.545923 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.545925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.545935 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.545990 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:03.546311 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.546997 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.547730 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.547939 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.548015 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.549002 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:03.549004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:03.549004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:03.549009 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:03.550375 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.550761 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.550762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.550763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.550811 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.551010 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.551011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.551011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.551103 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:03.551152 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.551316 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.551370 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:43:03.551573 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.545644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42639 (local address 127.31.63.67:35127)
0809 08:43:03.545797 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.545809 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.545873 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.545996 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.545996 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.546177 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:03.546226 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.546227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.547119 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.547120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.547642 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.547644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.548016 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.548017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.548085 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.548544 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:03.548549 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:03.548923 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:03.550451 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.550452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.550452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.550685 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.550875 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.550876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.550876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.550950 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.551165 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:03.551166 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:03.551216 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.551302 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.551421 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264983, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 08:43:03.570281 10719 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:03.564931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43892 (local address 127.31.63.126:42639)
0809 08:43:03.565000 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.565002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.565038 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:43:03.565060 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.565060 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.565070 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.565110 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:03.565797 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.566276 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.567025 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.567205 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.567256 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.570184 (+ 2928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43892: BlockingRecv error: recv got EOF from 127.0.0.1:43892 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264983, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:43:03.577653 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.571082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43902 (local address 127.31.63.126:42639)
0809 08:43:03.571141 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.571142 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.571185 (+ 43us) server_negotiation.cc:407] Connection header received
0809 08:43:03.571235 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.571235 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.571243 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.571302 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:03.571732 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.572167 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.572790 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.572957 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.573004 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.575037 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:03.575038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:03.575038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:03.575046 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:03.576496 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.576867 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.576869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.576869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.576928 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.577132 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.577133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.577133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.577241 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:03.577355 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.577451 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.577555 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:43:03.579356 10648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:43:03.580864 10648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:43:03.587503 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.583460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:37063 (local address 127.0.0.1:48758)
0809 08:43:03.583877 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.583887 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.583941 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.584176 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.584177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.584179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:03.584234 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.584236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.585087 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.585088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.586423 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.586427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.587182 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.587183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.587279 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.587295 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.587363 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.587422 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":71,"threads_started":1}
I20260809 08:43:03.587615 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.583478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43872 (local address 127.31.63.66:45429)
0809 08:43:03.583631 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.583637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.583650 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:43:03.583694 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.583697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.583715 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.584068 (+ 353us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:03.584133 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.585215 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.586470 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.587250 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.587339 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.587430 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.587458 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.587499 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260809 08:43:03.587813 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.583675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.31.63.67:37063)
0809 08:43:03.583875 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.583879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.583929 (+ 50us) server_negotiation.cc:407] Connection header received
0809 08:43:03.584016 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.584019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.584035 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.584109 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:03.584330 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.584996 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.586525 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.587078 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.587169 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.587257 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.587676 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.587718 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260809 08:43:03.587598 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.583269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:45429 (local address 127.0.0.1:43872)
0809 08:43:03.583434 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.583475 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.583538 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.583842 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.583846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.583849 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:03.583937 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.583939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.585336 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.585337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.586380 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.586383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.587339 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.587340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.587396 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.587399 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.587439 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.587469 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":4,"thread_start_us":57,"threads_started":1}
I20260809 08:43:03.587918 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.583398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.31.63.65:35265)
0809 08:43:03.583680 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.583685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.583694 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:43:03.583759 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.583761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.583774 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.584024 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:03.584096 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.585196 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.586608 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.587291 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.587641 (+ 350us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.587737 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.587766 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.587811 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":92,"threads_started":1}
I20260809 08:43:03.588142 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.583275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35265 (local address 127.0.0.1:51180)
0809 08:43:03.583640 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.583650 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.583689 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.583876 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.583876 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.583878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:03.583936 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.583938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.585292 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.585293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.586532 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.586534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.587389 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.587390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.587469 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.587473 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.587504 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.587542 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":22,"thread_start_us":69,"threads_started":1}
I20260809 08:43:03.588647 11034 tablet_service.cc:1467] Processing CreateTablet for tablet 2ba40a8bb1424df8b9bb88c26fa06bae (DEFAULT_TABLE table=test-table [id=6ad5b04634644426ad9e17907e90dc1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:03.588663 11237 tablet_service.cc:1467] Processing CreateTablet for tablet 2ba40a8bb1424df8b9bb88c26fa06bae (DEFAULT_TABLE table=test-table [id=6ad5b04634644426ad9e17907e90dc1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:03.588794 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 2ba40a8bb1424df8b9bb88c26fa06bae (DEFAULT_TABLE table=test-table [id=6ad5b04634644426ad9e17907e90dc1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:03.589010 11034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ba40a8bb1424df8b9bb88c26fa06bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:03.589051 11237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ba40a8bb1424df8b9bb88c26fa06bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:03.589079 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ba40a8bb1424df8b9bb88c26fa06bae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:03.592464 11375 tablet_bootstrap.cc:492] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: Bootstrap starting.
I20260809 08:43:03.592989 11377 tablet_bootstrap.cc:492] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: Bootstrap starting.
I20260809 08:43:03.593132 11379 tablet_bootstrap.cc:492] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: Bootstrap starting.
I20260809 08:43:03.593179 11375 tablet_bootstrap.cc:654] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:03.593494 11375 log.cc:824] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:03.593750 11377 tablet_bootstrap.cc:654] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:03.593750 11379 tablet_bootstrap.cc:654] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:03.594116 11379 log.cc:824] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:03.594106 11377 log.cc:824] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:03.594352 11375 tablet_bootstrap.cc:492] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: No bootstrap required, opened a new log
I20260809 08:43:03.594436 11375 ts_tablet_manager.cc:1397] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 08:43:03.594923 11379 tablet_bootstrap.cc:492] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: No bootstrap required, opened a new log
I20260809 08:43:03.595008 11379 ts_tablet_manager.cc:1397] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:43:03.595105 11377 tablet_bootstrap.cc:492] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: No bootstrap required, opened a new log
I20260809 08:43:03.595183 11377 ts_tablet_manager.cc:1397] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:43:03.596500 11379 raft_consensus.cc:348] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.596653 11379 raft_consensus.cc:374] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:03.596719 11379 raft_consensus.cc:729] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7d927b63ca04493b9363616bbc03660, State: Initialized, Role: FOLLOWER
I20260809 08:43:03.596684 11375 raft_consensus.cc:348] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.596858 11375 raft_consensus.cc:374] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:03.596848 11379 consensus_queue.cc:260] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.596949 11375 raft_consensus.cc:729] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29a7e8c2c414b968de49a8d00114007, State: Initialized, Role: FOLLOWER
I20260809 08:43:03.597081 11375 consensus_queue.cc:260] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.597123 11379 ts_tablet_manager.cc:1428] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:43:03.597272 10934 heartbeater.cc:503] Master 127.31.63.126:42639 was elected leader, sending a full tablet report...
I20260809 08:43:03.597359 11375 ts_tablet_manager.cc:1428] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:43:03.597249 11377 raft_consensus.cc:348] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.597405 11377 raft_consensus.cc:374] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:03.597409 11120 heartbeater.cc:503] Master 127.31.63.126:42639 was elected leader, sending a full tablet report...
I20260809 08:43:03.597440 11377 raft_consensus.cc:729] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f43232019c4e9e9ff3900c642028a3, State: Initialized, Role: FOLLOWER
I20260809 08:43:03.597613 11377 consensus_queue.cc:260] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.597932 11377 ts_tablet_manager.cc:1428] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:43:03.597962 11334 heartbeater.cc:503] Master 127.31.63.126:42639 was elected leader, sending a full tablet report...
W20260809 08:43:03.637490 11121 tablet.cc:2369] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:43:03.724050 10935 tablet.cc:2369] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:43:03.772231 11387 raft_consensus.cc:482] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:43:03.772372 11387 raft_consensus.cc:504] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.772784 11387 leader_election.cc:288] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265), 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063)
I20260809 08:43:03.775779 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.773088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:39963 (local address 127.31.63.65:35265)
0809 08:43:03.773135 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.773138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.773149 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:43:03.773184 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.773185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.773196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.773287 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:03.773484 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.774036 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.775030 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.775401 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.775457 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.775465 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.775665 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.775710 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:43:03.775804 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.772990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35265 (local address 127.31.63.66:39963)
0809 08:43:03.773059 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.773072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.773121 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.773308 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.773310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.773312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:03.773397 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.773399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.774120 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.774121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.774931 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.774933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.775489 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.775490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.775586 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.775590 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.775642 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.775675 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":1}
I20260809 08:43:03.775867 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.773088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:55415 (local address 127.31.63.67:37063)
0809 08:43:03.773143 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.773144 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.773305 (+ 161us) server_negotiation.cc:407] Connection header received
0809 08:43:03.773342 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.773343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.773354 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.773418 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:03.773627 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.774111 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.775072 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.775480 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.775555 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.775564 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.775743 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.775785 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:43:03.776046 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.772990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:37063 (local address 127.31.63.66:55415)
0809 08:43:03.773222 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.773233 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.773294 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.773449 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.773450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.773452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:03.773535 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.773537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.774194 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.774195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.774944 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.774945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.775568 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.775569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.775649 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.775653 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.775711 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.775747 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":43,"thread_start_us":125,"threads_started":1}
I20260809 08:43:03.776191 10863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ba40a8bb1424df8b9bb88c26fa06bae" candidate_uuid: "b29a7e8c2c414b968de49a8d00114007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d927b63ca04493b9363616bbc03660" is_pre_election: true
I20260809 08:43:03.776376 10863 raft_consensus.cc:2454] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b29a7e8c2c414b968de49a8d00114007 in term 0.
I20260809 08:43:03.776414 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ba40a8bb1424df8b9bb88c26fa06bae" candidate_uuid: "b29a7e8c2c414b968de49a8d00114007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f43232019c4e9e9ff3900c642028a3" is_pre_election: true
I20260809 08:43:03.776598 11265 raft_consensus.cc:2454] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b29a7e8c2c414b968de49a8d00114007 in term 0.
I20260809 08:43:03.776975 10972 leader_election.cc:302] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b29a7e8c2c414b968de49a8d00114007, f7d927b63ca04493b9363616bbc03660; no voters:
I20260809 08:43:03.777129 11387 raft_consensus.cc:2781] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:43:03.777190 11387 raft_consensus.cc:482] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:43:03.777209 11387 raft_consensus.cc:3037] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:03.778002 11387 raft_consensus.cc:504] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.778151 11387 leader_election.cc:288] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 election: Requested vote from peers f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265), 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063)
I20260809 08:43:03.778344 10863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ba40a8bb1424df8b9bb88c26fa06bae" candidate_uuid: "b29a7e8c2c414b968de49a8d00114007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d927b63ca04493b9363616bbc03660"
I20260809 08:43:03.778345 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ba40a8bb1424df8b9bb88c26fa06bae" candidate_uuid: "b29a7e8c2c414b968de49a8d00114007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f43232019c4e9e9ff3900c642028a3"
I20260809 08:43:03.778429 10863 raft_consensus.cc:3037] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:03.778430 11265 raft_consensus.cc:3037] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:03.779333 10863 raft_consensus.cc:2454] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b29a7e8c2c414b968de49a8d00114007 in term 1.
I20260809 08:43:03.779333 11265 raft_consensus.cc:2454] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b29a7e8c2c414b968de49a8d00114007 in term 1.
I20260809 08:43:03.779539 10971 leader_election.cc:302] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f43232019c4e9e9ff3900c642028a3, b29a7e8c2c414b968de49a8d00114007; no voters:
I20260809 08:43:03.779665 11387 raft_consensus.cc:2781] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:03.779824 11387 raft_consensus.cc:686] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 1 LEADER]: Becoming Leader. State: Replica: b29a7e8c2c414b968de49a8d00114007, State: Running, Role: LEADER
I20260809 08:43:03.779922 11387 consensus_queue.cc:237] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.780814 10648 catalog_manager.cc:5495] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 reported cstate change: term changed from 0 to 1, leader changed from <none> to b29a7e8c2c414b968de49a8d00114007 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "b29a7e8c2c414b968de49a8d00114007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } health_report { overall_health: UNKNOWN } } }
W20260809 08:43:03.792153 11335 tablet.cc:2369] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:43:03.813058 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.810730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43882 (local address 127.31.63.66:45429)
0809 08:43:03.810812 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.810813 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.810821 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:43:03.810863 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.810864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.810884 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.810949 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:43:03.811111 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.811596 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.812368 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.812552 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.812602 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.812760 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:43:03.812874 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:43:03.812935 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.812977 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.812996 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 08:43:03.814997 10863 raft_consensus.cc:1261] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Refusing update from remote peer b29a7e8c2c414b968de49a8d00114007: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:03.814997 11265 raft_consensus.cc:1261] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Refusing update from remote peer b29a7e8c2c414b968de49a8d00114007: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:03.815367 11387 consensus_queue.cc:1035] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:03.815546 11390 consensus_queue.cc:1035] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:03.818439 11396 mvcc.cc:205] Tried to move back new op lower bound from 7316541373703290880 to 7316541373563211776. Current Snapshot: MvccSnapshot[applied={T|T < 7316541373703290880}]
I20260809 08:43:03.818994 11397 mvcc.cc:205] Tried to move back new op lower bound from 7316541373703290880 to 7316541373563211776. Current Snapshot: MvccSnapshot[applied={T|T < 7316541373703290880 or (T in {7316541373703290880})}]
I20260809 08:43:03.819249 11399 mvcc.cc:205] Tried to move back new op lower bound from 7316541373703290880 to 7316541373563211776. Current Snapshot: MvccSnapshot[applied={T|T < 7316541373703290880}]
I20260809 08:43:03.820859 10649 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:43:03.820951 10649 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:43:03.821686 10649 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e: Sending DeleteTablet for 3 replicas of tablet 2ba40a8bb1424df8b9bb88c26fa06bae
I20260809 08:43:03.822013 10834 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ba40a8bb1424df8b9bb88c26fa06bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:03 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:51180
I20260809 08:43:03.822099 11034 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ba40a8bb1424df8b9bb88c26fa06bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:03 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:43872
I20260809 08:43:03.822149 11237 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ba40a8bb1424df8b9bb88c26fa06bae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:03 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:48758
I20260809 08:43:03.822352 11410 tablet_replica.cc:330] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: stopping tablet replica
I20260809 08:43:03.822485 11410 raft_consensus.cc:2229] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:43:03.822587 11410 raft_consensus.cc:2258] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:03.822575 10649 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:43:03.822721 10649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:43:03.823043 11410 ts_tablet_manager.cc:1905] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:03.824041 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 07fa6d848f5b484d8588f53443fc5179 (DEFAULT_TABLE table=test-table [id=9648964b261349c3abec62fd1ae5616d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:03.824112 11034 tablet_service.cc:1467] Processing CreateTablet for tablet 07fa6d848f5b484d8588f53443fc5179 (DEFAULT_TABLE table=test-table [id=9648964b261349c3abec62fd1ae5616d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:03.824182 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07fa6d848f5b484d8588f53443fc5179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:03.824221 11034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07fa6d848f5b484d8588f53443fc5179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:03.824525 11237 tablet_service.cc:1467] Processing CreateTablet for tablet 07fa6d848f5b484d8588f53443fc5179 (DEFAULT_TABLE table=test-table [id=9648964b261349c3abec62fd1ae5616d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:03.824653 11237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07fa6d848f5b484d8588f53443fc5179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:03.824956 11410 ts_tablet_manager.cc:1918] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:43:03.825026 11410 log.cc:1196] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/2ba40a8bb1424df8b9bb88c26fa06bae
I20260809 08:43:03.825304 11410 ts_tablet_manager.cc:1939] T 2ba40a8bb1424df8b9bb88c26fa06bae P b29a7e8c2c414b968de49a8d00114007: Deleting consensus metadata
I20260809 08:43:03.825651 11375 tablet_bootstrap.cc:492] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007: Bootstrap starting.
I20260809 08:43:03.826094 11375 tablet_bootstrap.cc:654] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:03.826433 11377 tablet_bootstrap.cc:492] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3: Bootstrap starting.
I20260809 08:43:03.826448 11379 tablet_bootstrap.cc:492] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660: Bootstrap starting.
I20260809 08:43:03.826828 11377 tablet_bootstrap.cc:654] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:03.826855 11379 tablet_bootstrap.cc:654] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:03.827345 10630 catalog_manager.cc:4841] TS b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429): tablet 2ba40a8bb1424df8b9bb88c26fa06bae (table test-table [id=6ad5b04634644426ad9e17907e90dc1b]) successfully deleted
I20260809 08:43:03.827498 11377 tablet_bootstrap.cc:492] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3: No bootstrap required, opened a new log
I20260809 08:43:03.827548 11377 ts_tablet_manager.cc:1397] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:03.827725 11377 raft_consensus.cc:348] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.827847 11377 raft_consensus.cc:374] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:03.827880 11377 raft_consensus.cc:729] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f43232019c4e9e9ff3900c642028a3, State: Initialized, Role: FOLLOWER
I20260809 08:43:03.827934 11377 consensus_queue.cc:260] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.827976 11375 tablet_bootstrap.cc:492] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007: No bootstrap required, opened a new log
I20260809 08:43:03.828020 11375 ts_tablet_manager.cc:1397] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 08:43:03.828022 11377 ts_tablet_manager.cc:1428] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:03.828131 11375 raft_consensus.cc:348] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.828176 11375 raft_consensus.cc:374] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:03.828188 11375 raft_consensus.cc:729] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29a7e8c2c414b968de49a8d00114007, State: Initialized, Role: FOLLOWER
I20260809 08:43:03.828258 11375 consensus_queue.cc:260] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.828348 11375 ts_tablet_manager.cc:1428] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:43:03.827724 11411 tablet_replica.cc:330] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: stopping tablet replica
I20260809 08:43:03.828567 11411 raft_consensus.cc:2229] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:43:03.828938 11411 raft_consensus.cc:2258] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:03.829246 11409 tablet_replica.cc:330] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: stopping tablet replica
I20260809 08:43:03.829368 11409 raft_consensus.cc:2229] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:43:03.829473 11409 raft_consensus.cc:2258] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:03.829927 11379 tablet_bootstrap.cc:492] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660: No bootstrap required, opened a new log
I20260809 08:43:03.829984 11379 ts_tablet_manager.cc:1397] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 08:43:03.830150 11379 raft_consensus.cc:348] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.830224 11379 raft_consensus.cc:374] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:03.830245 11379 raft_consensus.cc:729] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7d927b63ca04493b9363616bbc03660, State: Initialized, Role: FOLLOWER
I20260809 08:43:03.830310 11379 consensus_queue.cc:260] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.830415 11379 ts_tablet_manager.cc:1428] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:43:03.831053 11411 ts_tablet_manager.cc:1905] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:03.832604 11411 ts_tablet_manager.cc:1918] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:43:03.832685 11411 log.cc:1196] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/2ba40a8bb1424df8b9bb88c26fa06bae
I20260809 08:43:03.832983 11411 ts_tablet_manager.cc:1939] T 2ba40a8bb1424df8b9bb88c26fa06bae P 41f43232019c4e9e9ff3900c642028a3: Deleting consensus metadata
I20260809 08:43:03.833494 10630 catalog_manager.cc:4841] TS 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063): tablet 2ba40a8bb1424df8b9bb88c26fa06bae (table test-table [id=6ad5b04634644426ad9e17907e90dc1b]) successfully deleted
I20260809 08:43:03.833652 11409 ts_tablet_manager.cc:1905] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:03.835556 11409 ts_tablet_manager.cc:1918] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:43:03.835632 11409 log.cc:1196] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/2ba40a8bb1424df8b9bb88c26fa06bae
I20260809 08:43:03.835898 11409 ts_tablet_manager.cc:1939] T 2ba40a8bb1424df8b9bb88c26fa06bae P f7d927b63ca04493b9363616bbc03660: Deleting consensus metadata
I20260809 08:43:03.836376 10631 catalog_manager.cc:4841] TS f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265): tablet 2ba40a8bb1424df8b9bb88c26fa06bae (table test-table [id=6ad5b04634644426ad9e17907e90dc1b]) successfully deleted
I20260809 08:43:03.918797 11387 raft_consensus.cc:482] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:43:03.918916 11387 raft_consensus.cc:504] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.919143 11387 leader_election.cc:288] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265), 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063)
I20260809 08:43:03.919396 10863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07fa6d848f5b484d8588f53443fc5179" candidate_uuid: "b29a7e8c2c414b968de49a8d00114007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d927b63ca04493b9363616bbc03660" is_pre_election: true
I20260809 08:43:03.919404 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07fa6d848f5b484d8588f53443fc5179" candidate_uuid: "b29a7e8c2c414b968de49a8d00114007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f43232019c4e9e9ff3900c642028a3" is_pre_election: true
I20260809 08:43:03.919500 10863 raft_consensus.cc:2454] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b29a7e8c2c414b968de49a8d00114007 in term 0.
I20260809 08:43:03.919523 11265 raft_consensus.cc:2454] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b29a7e8c2c414b968de49a8d00114007 in term 0.
I20260809 08:43:03.919695 10971 leader_election.cc:302] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f43232019c4e9e9ff3900c642028a3, b29a7e8c2c414b968de49a8d00114007; no voters:
I20260809 08:43:03.919811 11387 raft_consensus.cc:2781] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:43:03.919859 11387 raft_consensus.cc:482] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:43:03.919883 11387 raft_consensus.cc:3037] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:03.920909 11387 raft_consensus.cc:504] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.921066 11387 leader_election.cc:288] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 election: Requested vote from peers f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265), 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063)
I20260809 08:43:03.921224 10863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07fa6d848f5b484d8588f53443fc5179" candidate_uuid: "b29a7e8c2c414b968de49a8d00114007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d927b63ca04493b9363616bbc03660"
I20260809 08:43:03.921331 10863 raft_consensus.cc:3037] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:03.921320 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07fa6d848f5b484d8588f53443fc5179" candidate_uuid: "b29a7e8c2c414b968de49a8d00114007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f43232019c4e9e9ff3900c642028a3"
I20260809 08:43:03.921402 11265 raft_consensus.cc:3037] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:03.922070 10863 raft_consensus.cc:2454] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b29a7e8c2c414b968de49a8d00114007 in term 1.
I20260809 08:43:03.922253 10972 leader_election.cc:302] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b29a7e8c2c414b968de49a8d00114007, f7d927b63ca04493b9363616bbc03660; no voters:
I20260809 08:43:03.922247 11265 raft_consensus.cc:2454] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b29a7e8c2c414b968de49a8d00114007 in term 1.
I20260809 08:43:03.922428 11387 raft_consensus.cc:2781] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:03.922485 11387 raft_consensus.cc:686] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 1 LEADER]: Becoming Leader. State: Replica: b29a7e8c2c414b968de49a8d00114007, State: Running, Role: LEADER
I20260809 08:43:03.922673 11387 consensus_queue.cc:237] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.923334 10649 catalog_manager.cc:5495] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 reported cstate change: term changed from 0 to 1, leader changed from <none> to b29a7e8c2c414b968de49a8d00114007 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "b29a7e8c2c414b968de49a8d00114007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } health_report { overall_health: UNKNOWN } } }
Aug 09 08:43:03 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264982, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.126@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:43:03.973284 11424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.966285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42639 (local address 127.0.0.1:43910)
0809 08:43:03.966500 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:03.966511 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:03.966580 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:03.966732 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:03.966732 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:03.967054 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:03.967105 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.967107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.967773 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.967774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.968299 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:03.968300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.968641 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.968642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:03.968683 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.969221 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:03.969226 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:03.970688 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:03.972292 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.972299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.972301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.972490 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.972687 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:03.972687 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:03.972687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:03.972770 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:03.972992 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:03.972999 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:03.973042 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 08:43:03.973095 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:43:03.973188 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":125,"threads_started":1}
I20260809 08:43:03.973328 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:03.966387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43910 (local address 127.31.63.126:42639)
0809 08:43:03.966447 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:03.966449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:03.966584 (+ 135us) server_negotiation.cc:407] Connection header received
0809 08:43:03.966627 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:03.966628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:03.966638 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:03.966720 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:03.967181 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.967694 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.968381 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:03.968570 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:03.968629 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:03.970778 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:03.970779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:03.970779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:03.970786 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:03.972224 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.972559 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.972560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.972561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.972622 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:03.972826 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:03.972827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:03.972827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:03.972935 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:03.972985 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:03.973120 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 08:43:03.973233 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":32}
I20260809 08:43:03.975723 10649 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:43910:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 08:43:03.975963 10649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:43:03.977687 10834 tablet_service.cc:1467] Processing CreateTablet for tablet e1688a4ae35b4a8fa53dcea5043d9892 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cd086bb97a147a5aabd5611207ca2cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:43:03.977717 11034 tablet_service.cc:1467] Processing CreateTablet for tablet e1688a4ae35b4a8fa53dcea5043d9892 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cd086bb97a147a5aabd5611207ca2cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:43:03.977735 11237 tablet_service.cc:1467] Processing CreateTablet for tablet e1688a4ae35b4a8fa53dcea5043d9892 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2cd086bb97a147a5aabd5611207ca2cd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:43:03.977890 11237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1688a4ae35b4a8fa53dcea5043d9892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:03.977890 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1688a4ae35b4a8fa53dcea5043d9892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:03.977912 11034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1688a4ae35b4a8fa53dcea5043d9892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:03.979278 11375 tablet_bootstrap.cc:492] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: Bootstrap starting.
I20260809 08:43:03.979472 11379 tablet_bootstrap.cc:492] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: Bootstrap starting.
I20260809 08:43:03.979699 11375 tablet_bootstrap.cc:654] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:03.979933 11379 tablet_bootstrap.cc:654] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:03.980324 11375 tablet_bootstrap.cc:492] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: No bootstrap required, opened a new log
I20260809 08:43:03.980365 11377 tablet_bootstrap.cc:492] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: Bootstrap starting.
I20260809 08:43:03.980396 11375 ts_tablet_manager.cc:1397] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:43:03.980530 11375 raft_consensus.cc:348] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.980623 11375 raft_consensus.cc:374] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:03.980650 11375 raft_consensus.cc:729] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29a7e8c2c414b968de49a8d00114007, State: Initialized, Role: FOLLOWER
I20260809 08:43:03.980702 11375 consensus_queue.cc:260] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.980808 11377 tablet_bootstrap.cc:654] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:03.980829 11375 ts_tablet_manager.cc:1428] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:43:03.980845 11402 raft_consensus.cc:482] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:43:03.980912 11402 raft_consensus.cc:504] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.981025 11387 tablet_replica.cc:439] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } } }
I20260809 08:43:03.981078 11402 leader_election.cc:288] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265), 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063)
I20260809 08:43:03.981108 11387 tablet_replica.cc:442] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:43:03.981274 10863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1688a4ae35b4a8fa53dcea5043d9892" candidate_uuid: "b29a7e8c2c414b968de49a8d00114007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d927b63ca04493b9363616bbc03660" is_pre_election: true
I20260809 08:43:03.981484 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1688a4ae35b4a8fa53dcea5043d9892" candidate_uuid: "b29a7e8c2c414b968de49a8d00114007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f43232019c4e9e9ff3900c642028a3" is_pre_election: true
W20260809 08:43:03.981756 10972 leader_election.cc:341] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265): Illegal state: must be running to vote when last-logged opid is not known
I20260809 08:43:03.981837 11379 tablet_bootstrap.cc:492] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: No bootstrap required, opened a new log
W20260809 08:43:03.981851 10971 leader_election.cc:341] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063): Illegal state: must be running to vote when last-logged opid is not known
I20260809 08:43:03.981894 10971 leader_election.cc:302] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b29a7e8c2c414b968de49a8d00114007; no voters: 41f43232019c4e9e9ff3900c642028a3, f7d927b63ca04493b9363616bbc03660
I20260809 08:43:03.981890 11379 ts_tablet_manager.cc:1397] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:43:03.981954 11387 raft_consensus.cc:2726] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 08:43:03.982057 11379 raft_consensus.cc:348] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.982138 11379 raft_consensus.cc:374] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:03.982178 11379 raft_consensus.cc:729] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7d927b63ca04493b9363616bbc03660, State: Initialized, Role: FOLLOWER
I20260809 08:43:03.982239 11379 consensus_queue.cc:260] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.982350 11379 ts_tablet_manager.cc:1428] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:03.982451 11386 tablet_replica.cc:439] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } } }
I20260809 08:43:03.982553 11386 tablet_replica.cc:442] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:43:03.982849 11377 tablet_bootstrap.cc:492] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: No bootstrap required, opened a new log
I20260809 08:43:03.982901 11377 ts_tablet_manager.cc:1397] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 08:43:03.983072 11377 raft_consensus.cc:348] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.983145 11377 raft_consensus.cc:374] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:03.983160 11377 raft_consensus.cc:729] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f43232019c4e9e9ff3900c642028a3, State: Initialized, Role: FOLLOWER
I20260809 08:43:03.983227 11377 consensus_queue.cc:260] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:03.983332 11377 ts_tablet_manager.cc:1428] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:03.983405 11388 tablet_replica.cc:439] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } } }
I20260809 08:43:03.983559 11388 tablet_replica.cc:442] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:43:04.033880 11388 raft_consensus.cc:482] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:43:04.033998 11388 raft_consensus.cc:504] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.034380 11388 leader_election.cc:288] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265), b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429)
I20260809 08:43:04.038162 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.034630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:49941 (local address 127.31.63.65:35265)
0809 08:43:04.034684 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.034686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.034698 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:43:04.034806 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.034807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.034819 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.034879 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:04.035097 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.035705 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.037184 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.037748 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.037829 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.037839 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.038042 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.038087 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:43:04.038162 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.034750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:40859 (local address 127.31.63.66:45429)
0809 08:43:04.034804 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.034812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.034820 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:43:04.034929 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.034930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.034944 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.035013 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:04.035221 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.035907 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.037139 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.037749 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.037829 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.037839 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.038041 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.038086 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:43:04.038225 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.034554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35265 (local address 127.31.63.67:49941)
0809 08:43:04.034628 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:04.034641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:04.034702 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:04.034912 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:04.034913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:04.034915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:04.035006 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.035008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.035789 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.035790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.037069 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.037073 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.037842 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.037843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.037945 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.037950 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:43:04.038017 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:43:04.038067 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":19}
I20260809 08:43:04.038237 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.034602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:45429 (local address 127.31.63.67:40859)
0809 08:43:04.034760 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:04.034770 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:04.034829 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:04.035024 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:04.035025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:04.035027 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:04.035136 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.035138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.035995 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.035996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.037067 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.037071 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.037843 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.037843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.037945 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.037952 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:43:04.038017 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:43:04.038067 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":9,"thread_start_us":104,"threads_started":1}
I20260809 08:43:04.038638 11057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1688a4ae35b4a8fa53dcea5043d9892" candidate_uuid: "41f43232019c4e9e9ff3900c642028a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b29a7e8c2c414b968de49a8d00114007" is_pre_election: true
I20260809 08:43:04.038686 10863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1688a4ae35b4a8fa53dcea5043d9892" candidate_uuid: "41f43232019c4e9e9ff3900c642028a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d927b63ca04493b9363616bbc03660" is_pre_election: true
I20260809 08:43:04.038792 10863 raft_consensus.cc:2454] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41f43232019c4e9e9ff3900c642028a3 in term 0.
I20260809 08:43:04.038802 11057 raft_consensus.cc:2454] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41f43232019c4e9e9ff3900c642028a3 in term 0.
I20260809 08:43:04.039036 11163 leader_election.cc:302] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f43232019c4e9e9ff3900c642028a3, b29a7e8c2c414b968de49a8d00114007; no voters:
I20260809 08:43:04.039232 11388 raft_consensus.cc:2781] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:43:04.039295 11388 raft_consensus.cc:482] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:43:04.039327 11388 raft_consensus.cc:3037] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:04.040378 11388 raft_consensus.cc:504] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.040540 11388 leader_election.cc:288] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [CANDIDATE]: Term 1 election: Requested vote from peers f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265), b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429)
I20260809 08:43:04.040711 10863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1688a4ae35b4a8fa53dcea5043d9892" candidate_uuid: "41f43232019c4e9e9ff3900c642028a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d927b63ca04493b9363616bbc03660"
I20260809 08:43:04.040735 11057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1688a4ae35b4a8fa53dcea5043d9892" candidate_uuid: "41f43232019c4e9e9ff3900c642028a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b29a7e8c2c414b968de49a8d00114007"
I20260809 08:43:04.040812 10863 raft_consensus.cc:3037] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:04.040829 11057 raft_consensus.cc:3037] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:04.041410 10863 raft_consensus.cc:2454] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41f43232019c4e9e9ff3900c642028a3 in term 1.
I20260809 08:43:04.041653 11057 raft_consensus.cc:2454] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41f43232019c4e9e9ff3900c642028a3 in term 1.
I20260809 08:43:04.041674 11165 leader_election.cc:302] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f43232019c4e9e9ff3900c642028a3, f7d927b63ca04493b9363616bbc03660; no voters:
I20260809 08:43:04.041795 11388 raft_consensus.cc:2781] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:04.041932 11388 raft_consensus.cc:686] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [term 1 LEADER]: Becoming Leader. State: Replica: 41f43232019c4e9e9ff3900c642028a3, State: Running, Role: LEADER
I20260809 08:43:04.042006 11388 consensus_queue.cc:237] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.042677 11430 tablet_replica.cc:439] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: TxnStatusTablet state changed. Reason: New leader 41f43232019c4e9e9ff3900c642028a3. Latest consensus state: current_term: 1 leader_uuid: "41f43232019c4e9e9ff3900c642028a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } } }
I20260809 08:43:04.042774 11430 tablet_replica.cc:442] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:43:04.042764 10649 catalog_manager.cc:5495] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67). New cstate: current_term: 1 leader_uuid: "41f43232019c4e9e9ff3900c642028a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } health_report { overall_health: HEALTHY } } }
I20260809 08:43:04.042997 11432 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 08:43:04.043131 11432 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 08:43:04.057263 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.054505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.31.63.67:37063)
0809 08:43:04.054571 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.054573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.054583 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:43:04.054677 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.054678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.054696 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.054757 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:43:04.054923 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.055423 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.056520 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.056697 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.056744 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.056961 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.057057 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.057105 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.057187 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.057205 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:43:04.057324 11424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.054445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:37063 (local address 127.0.0.1:48766)
0809 08:43:04.054533 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:04.054545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:04.054590 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:04.054791 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:04.054793 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:04.054794 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:43:04.054855 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.054857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.055499 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.055500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.056433 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.056436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.056773 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.056774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.056845 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.056864 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.057117 (+ 253us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.057125 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 08:43:04.057186 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:43:04.057236 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 08:43:04.058590 10863 raft_consensus.cc:1261] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Refusing update from remote peer 41f43232019c4e9e9ff3900c642028a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:04.058681 11057 raft_consensus.cc:1261] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Refusing update from remote peer 41f43232019c4e9e9ff3900c642028a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:04.058884 11388 consensus_queue.cc:1035] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:04.059011 11430 consensus_queue.cc:1035] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:04.059739 11387 tablet_replica.cc:439] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: TxnStatusTablet state changed. Reason: New leader 41f43232019c4e9e9ff3900c642028a3. Latest consensus state: current_term: 1 leader_uuid: "41f43232019c4e9e9ff3900c642028a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } } }
I20260809 08:43:04.059860 11387 tablet_replica.cc:442] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:43:04.059988 11386 tablet_replica.cc:439] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: TxnStatusTablet state changed. Reason: New leader 41f43232019c4e9e9ff3900c642028a3. Latest consensus state: current_term: 1 leader_uuid: "41f43232019c4e9e9ff3900c642028a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } } }
I20260809 08:43:04.060094 11386 tablet_replica.cc:442] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:43:04.060420 11388 tablet_replica.cc:439] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41f43232019c4e9e9ff3900c642028a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } } }
I20260809 08:43:04.060510 11388 tablet_replica.cc:442] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:43:04.061275 11388 tablet_replica.cc:439] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41f43232019c4e9e9ff3900c642028a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } } }
I20260809 08:43:04.061350 11388 tablet_replica.cc:442] T e1688a4ae35b4a8fa53dcea5043d9892 P 41f43232019c4e9e9ff3900c642028a3: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:43:04.061293 11386 tablet_replica.cc:439] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41f43232019c4e9e9ff3900c642028a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } } }
I20260809 08:43:04.061384 11386 tablet_replica.cc:442] T e1688a4ae35b4a8fa53dcea5043d9892 P f7d927b63ca04493b9363616bbc03660: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:43:04.062251 11387 tablet_replica.cc:439] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41f43232019c4e9e9ff3900c642028a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } } }
I20260809 08:43:04.062340 11387 tablet_replica.cc:442] T e1688a4ae35b4a8fa53dcea5043d9892 P b29a7e8c2c414b968de49a8d00114007: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:43:04.066669 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.064103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:35715 (local address 127.31.63.67:37063)
0809 08:43:04.064145 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.064147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.064298 (+ 151us) server_negotiation.cc:407] Connection header received
0809 08:43:04.064369 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.064370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.064379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.064419 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:43:04.064540 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.064995 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.065927 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.066130 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.066233 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.066425 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.066463 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.066523 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.066580 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.066602 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:43:04.066705 11440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.064009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:37063 (local address 127.31.63.66:35715)
0809 08:43:04.064235 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:04.064246 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:04.064312 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:04.064432 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:04.064432 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:04.064434 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:43:04.064487 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.064489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.065083 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.065085 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.065842 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.065845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.066234 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.066235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.066311 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.066326 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.066527 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.066530 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:43:04.066577 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 08:43:04.066626 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":108,"threads_started":1}
I20260809 08:43:04.068971 11265 raft_consensus.cc:1261] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Refusing update from remote peer b29a7e8c2c414b968de49a8d00114007: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:04.069026 10863 raft_consensus.cc:1261] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Refusing update from remote peer b29a7e8c2c414b968de49a8d00114007: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:04.069198 11387 consensus_queue.cc:1035] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:04.069291 11402 consensus_queue.cc:1035] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:04.079391 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.075486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:39317 (local address 127.31.63.66:45429)
0809 08:43:04.075543 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.075545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.075553 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:43:04.075646 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.075647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.075656 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.075702 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:43:04.075844 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.076302 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.078146 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.078450 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.078494 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.079137 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.079180 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.079234 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.079294 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.079317 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:43:04.079432 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.075270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:45429 (local address 127.31.63.67:39317)
0809 08:43:04.075461 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:04.075474 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:04.075578 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:04.075706 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:04.075706 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:04.075708 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:43:04.075777 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.075779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.076393 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.076394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.078001 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.078004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.078726 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.078727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.079010 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.079030 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.079241 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.079244 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:43:04.079290 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 08:43:04.079338 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
I20260809 08:43:04.086593 10649 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:43:04.086686 10649 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:43:04.087248 10649 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e: Sending DeleteTablet for 3 replicas of tablet 07fa6d848f5b484d8588f53443fc5179
I20260809 08:43:04.087522 11237 tablet_service.cc:1514] Processing DeleteTablet for tablet 07fa6d848f5b484d8588f53443fc5179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:04 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:48758
I20260809 08:43:04.087540 10834 tablet_service.cc:1514] Processing DeleteTablet for tablet 07fa6d848f5b484d8588f53443fc5179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:04 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:51180
I20260809 08:43:04.087631 11411 tablet_replica.cc:330] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3: stopping tablet replica
I20260809 08:43:04.087668 11034 tablet_service.cc:1514] Processing DeleteTablet for tablet 07fa6d848f5b484d8588f53443fc5179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:04 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:43872
I20260809 08:43:04.087698 11411 raft_consensus.cc:2229] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:43:04.087735 11411 raft_consensus.cc:2258] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:04.087754 11410 tablet_replica.cc:330] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007: stopping tablet replica
I20260809 08:43:04.087807 11410 raft_consensus.cc:2229] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:43:04.087814 11409 tablet_replica.cc:330] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660: stopping tablet replica
I20260809 08:43:04.087862 11409 raft_consensus.cc:2229] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:43:04.087870 11410 raft_consensus.cc:2258] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:04.087890 11409 raft_consensus.cc:2258] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:04.088143 11410 ts_tablet_manager.cc:1905] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:04.088224 11409 ts_tablet_manager.cc:1905] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:04.088436 11411 ts_tablet_manager.cc:1905] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:04.089897 11410 ts_tablet_manager.cc:1918] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:43:04.089896 11409 ts_tablet_manager.cc:1918] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:43:04.089951 11410 log.cc:1196] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/07fa6d848f5b484d8588f53443fc5179
I20260809 08:43:04.089963 11409 log.cc:1196] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/07fa6d848f5b484d8588f53443fc5179
I20260809 08:43:04.090178 11409 ts_tablet_manager.cc:1939] T 07fa6d848f5b484d8588f53443fc5179 P f7d927b63ca04493b9363616bbc03660: Deleting consensus metadata
I20260809 08:43:04.090179 11410 ts_tablet_manager.cc:1939] T 07fa6d848f5b484d8588f53443fc5179 P b29a7e8c2c414b968de49a8d00114007: Deleting consensus metadata
I20260809 08:43:04.090394 11411 ts_tablet_manager.cc:1918] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:43:04.090453 11411 log.cc:1196] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/07fa6d848f5b484d8588f53443fc5179
I20260809 08:43:04.090502 10631 catalog_manager.cc:4841] TS f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265): tablet 07fa6d848f5b484d8588f53443fc5179 (table test-table [id=9648964b261349c3abec62fd1ae5616d]) successfully deleted
I20260809 08:43:04.090675 11411 ts_tablet_manager.cc:1939] T 07fa6d848f5b484d8588f53443fc5179 P 41f43232019c4e9e9ff3900c642028a3: Deleting consensus metadata
I20260809 08:43:04.090663 10630 catalog_manager.cc:4841] TS b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429): tablet 07fa6d848f5b484d8588f53443fc5179 (table test-table [id=9648964b261349c3abec62fd1ae5616d]) successfully deleted
I20260809 08:43:04.090972 10630 catalog_manager.cc:4841] TS 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063): tablet 07fa6d848f5b484d8588f53443fc5179 (table test-table [id=9648964b261349c3abec62fd1ae5616d]) successfully deleted
Aug 09 08:43:04 dist-test-slave-gkw7 krb5kdc[10586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:43:04 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:04 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 08:43:04.115777 10719 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:04.109335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.31.63.126:42639)
0809 08:43:04.109425 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.109428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.109622 (+ 194us) server_negotiation.cc:407] Connection header received
0809 08:43:04.109732 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.109734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.109752 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.109826 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:04.110460 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.111275 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.112065 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.112391 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.112470 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.115688 (+ 3218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43924: BlockingRecv error: recv got EOF from 127.0.0.1:43924 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 08:43:04 dist-test-slave-gkw7 krb5kdc[10586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM
I20260809 08:43:04.124867 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.116688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43936 (local address 127.31.63.126:42639)
0809 08:43:04.116738 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.116740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.116762 (+ 22us) server_negotiation.cc:407] Connection header received
0809 08:43:04.116862 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.116864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.116872 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.116919 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:04.117514 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.118104 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.119152 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.119353 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.119411 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.121985 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:04.121986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:04.121987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:04.122002 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:04.123534 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:04.123950 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:04.123951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:04.123952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:04.124045 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:04.124265 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:04.124266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:04.124267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:04.124393 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:04.124502 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.124606 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.124696 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:43:04.126732 10649 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:43:04.126952 10649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:43:04.128475 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 55696b0cf71a4c23a1829b7b377878fe (DEFAULT_TABLE table=test-table [id=9b3d98f4904c484893c511b6a443f889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:04.128516 11034 tablet_service.cc:1467] Processing CreateTablet for tablet 55696b0cf71a4c23a1829b7b377878fe (DEFAULT_TABLE table=test-table [id=9b3d98f4904c484893c511b6a443f889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:04.128580 11237 tablet_service.cc:1467] Processing CreateTablet for tablet 55696b0cf71a4c23a1829b7b377878fe (DEFAULT_TABLE table=test-table [id=9b3d98f4904c484893c511b6a443f889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:04.128635 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55696b0cf71a4c23a1829b7b377878fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:04.128662 11034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55696b0cf71a4c23a1829b7b377878fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:04.128688 11237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55696b0cf71a4c23a1829b7b377878fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:04.130111 11377 tablet_bootstrap.cc:492] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3: Bootstrap starting.
I20260809 08:43:04.130111 11375 tablet_bootstrap.cc:492] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007: Bootstrap starting.
I20260809 08:43:04.130180 11379 tablet_bootstrap.cc:492] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660: Bootstrap starting.
I20260809 08:43:04.130481 11375 tablet_bootstrap.cc:654] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:04.130481 11377 tablet_bootstrap.cc:654] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:04.130615 11379 tablet_bootstrap.cc:654] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:04.131028 11375 tablet_bootstrap.cc:492] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007: No bootstrap required, opened a new log
I20260809 08:43:04.131074 11375 ts_tablet_manager.cc:1397] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:43:04.131186 11375 raft_consensus.cc:348] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.131285 11375 raft_consensus.cc:374] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:04.131318 11375 raft_consensus.cc:729] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29a7e8c2c414b968de49a8d00114007, State: Initialized, Role: FOLLOWER
I20260809 08:43:04.131371 11375 consensus_queue.cc:260] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.131474 11375 ts_tablet_manager.cc:1428] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:43:04.131532 11379 tablet_bootstrap.cc:492] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660: No bootstrap required, opened a new log
I20260809 08:43:04.131593 11379 ts_tablet_manager.cc:1397] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:04.131789 11377 tablet_bootstrap.cc:492] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3: No bootstrap required, opened a new log
I20260809 08:43:04.131749 11379 raft_consensus.cc:348] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.131837 11379 raft_consensus.cc:374] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:04.131860 11379 raft_consensus.cc:729] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7d927b63ca04493b9363616bbc03660, State: Initialized, Role: FOLLOWER
I20260809 08:43:04.131865 11377 ts_tablet_manager.cc:1397] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:43:04.131906 11379 consensus_queue.cc:260] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.131999 11379 ts_tablet_manager.cc:1428] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:04.132030 11377 raft_consensus.cc:348] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.132109 11377 raft_consensus.cc:374] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:04.132136 11377 raft_consensus.cc:729] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f43232019c4e9e9ff3900c642028a3, State: Initialized, Role: FOLLOWER
I20260809 08:43:04.132185 11377 consensus_queue.cc:260] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.132272 11377 ts_tablet_manager.cc:1428] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:43:04.276350 11430 raft_consensus.cc:482] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:43:04.276492 11430 raft_consensus.cc:504] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.276708 11430 leader_election.cc:288] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265), b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429)
I20260809 08:43:04.276981 11057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55696b0cf71a4c23a1829b7b377878fe" candidate_uuid: "41f43232019c4e9e9ff3900c642028a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b29a7e8c2c414b968de49a8d00114007" is_pre_election: true
I20260809 08:43:04.277073 11057 raft_consensus.cc:2454] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41f43232019c4e9e9ff3900c642028a3 in term 0.
I20260809 08:43:04.276989 10863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55696b0cf71a4c23a1829b7b377878fe" candidate_uuid: "41f43232019c4e9e9ff3900c642028a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d927b63ca04493b9363616bbc03660" is_pre_election: true
I20260809 08:43:04.277117 10863 raft_consensus.cc:2454] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41f43232019c4e9e9ff3900c642028a3 in term 0.
I20260809 08:43:04.277266 11163 leader_election.cc:302] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f43232019c4e9e9ff3900c642028a3, b29a7e8c2c414b968de49a8d00114007; no voters:
I20260809 08:43:04.277374 11430 raft_consensus.cc:2781] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:43:04.277436 11430 raft_consensus.cc:482] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:43:04.277458 11430 raft_consensus.cc:3037] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:04.278368 11430 raft_consensus.cc:504] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.278513 11430 leader_election.cc:288] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [CANDIDATE]: Term 1 election: Requested vote from peers f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265), b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429)
I20260809 08:43:04.278690 11057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55696b0cf71a4c23a1829b7b377878fe" candidate_uuid: "41f43232019c4e9e9ff3900c642028a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b29a7e8c2c414b968de49a8d00114007"
I20260809 08:43:04.278692 10863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55696b0cf71a4c23a1829b7b377878fe" candidate_uuid: "41f43232019c4e9e9ff3900c642028a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d927b63ca04493b9363616bbc03660"
I20260809 08:43:04.278772 11057 raft_consensus.cc:3037] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:04.278781 10863 raft_consensus.cc:3037] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:04.279564 11057 raft_consensus.cc:2454] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41f43232019c4e9e9ff3900c642028a3 in term 1.
I20260809 08:43:04.279565 10863 raft_consensus.cc:2454] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41f43232019c4e9e9ff3900c642028a3 in term 1.
I20260809 08:43:04.279764 11163 leader_election.cc:302] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f43232019c4e9e9ff3900c642028a3, b29a7e8c2c414b968de49a8d00114007; no voters:
I20260809 08:43:04.279889 11430 raft_consensus.cc:2781] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:04.279939 11430 raft_consensus.cc:686] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 1 LEADER]: Becoming Leader. State: Replica: 41f43232019c4e9e9ff3900c642028a3, State: Running, Role: LEADER
I20260809 08:43:04.279995 11430 consensus_queue.cc:237] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.280618 10647 catalog_manager.cc:5495] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67). New cstate: current_term: 1 leader_uuid: "41f43232019c4e9e9ff3900c642028a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } health_report { overall_health: HEALTHY } } }
I20260809 08:43:04.318059 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.315845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.31.63.67:37063)
0809 08:43:04.315912 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.315914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.315923 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:43:04.315949 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.315949 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.315958 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.316014 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:43:04.316138 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.316632 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.317437 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.317676 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.317734 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.317863 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.317893 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.317933 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.317981 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.317997 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:43:04.319047 10863 raft_consensus.cc:1261] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Refusing update from remote peer 41f43232019c4e9e9ff3900c642028a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:04.319049 11057 raft_consensus.cc:1261] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Refusing update from remote peer 41f43232019c4e9e9ff3900c642028a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:04.319329 11442 consensus_queue.cc:1035] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:04.319463 11430 consensus_queue.cc:1035] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:04.322832 10647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:43:04.322933 10647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:43:04.323601 10647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e: Sending DeleteTablet for 3 replicas of tablet 55696b0cf71a4c23a1829b7b377878fe
I20260809 08:43:04.323887 11237 tablet_service.cc:1514] Processing DeleteTablet for tablet 55696b0cf71a4c23a1829b7b377878fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:04 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:48758
I20260809 08:43:04.323894 11034 tablet_service.cc:1514] Processing DeleteTablet for tablet 55696b0cf71a4c23a1829b7b377878fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:04 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:43872
I20260809 08:43:04.323925 10834 tablet_service.cc:1514] Processing DeleteTablet for tablet 55696b0cf71a4c23a1829b7b377878fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:04 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:51180
I20260809 08:43:04.323992 11410 tablet_replica.cc:330] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007: stopping tablet replica
I20260809 08:43:04.324028 11409 tablet_replica.cc:330] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660: stopping tablet replica
I20260809 08:43:04.324035 11411 tablet_replica.cc:330] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3: stopping tablet replica
I20260809 08:43:04.324046 11410 raft_consensus.cc:2229] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:43:04.324079 11409 raft_consensus.cc:2229] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:43:04.324090 11410 raft_consensus.cc:2258] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:04.324097 11411 raft_consensus.cc:2229] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:43:04.324121 11409 raft_consensus.cc:2258] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:04.324203 11411 raft_consensus.cc:2258] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:04.324522 11411 ts_tablet_manager.cc:1905] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:04.324604 11409 ts_tablet_manager.cc:1905] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:04.324697 10647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:43:04.324852 10647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:43:04.326177 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 4630ff5ab1d7477781a5812940c70a6a (DEFAULT_TABLE table=test-table [id=7d5bd2b6362b4fed9093e6626b695af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:04.326257 11411 ts_tablet_manager.cc:1918] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:43:04.326257 11034 tablet_service.cc:1467] Processing CreateTablet for tablet 4630ff5ab1d7477781a5812940c70a6a (DEFAULT_TABLE table=test-table [id=7d5bd2b6362b4fed9093e6626b695af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:04.326293 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4630ff5ab1d7477781a5812940c70a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:04.326314 11411 log.cc:1196] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/55696b0cf71a4c23a1829b7b377878fe
I20260809 08:43:04.326377 11034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4630ff5ab1d7477781a5812940c70a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:04.326543 11411 ts_tablet_manager.cc:1939] T 55696b0cf71a4c23a1829b7b377878fe P 41f43232019c4e9e9ff3900c642028a3: Deleting consensus metadata
I20260809 08:43:04.327502 11379 tablet_bootstrap.cc:492] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660: Bootstrap starting.
I20260809 08:43:04.327752 10630 catalog_manager.cc:4841] TS 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063): tablet 55696b0cf71a4c23a1829b7b377878fe (table test-table [id=9b3d98f4904c484893c511b6a443f889]) successfully deleted
I20260809 08:43:04.327924 11237 tablet_service.cc:1467] Processing CreateTablet for tablet 4630ff5ab1d7477781a5812940c70a6a (DEFAULT_TABLE table=test-table [id=7d5bd2b6362b4fed9093e6626b695af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:04.328025 11237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4630ff5ab1d7477781a5812940c70a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:04.328078 11410 ts_tablet_manager.cc:1905] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:04.328188 11379 tablet_bootstrap.cc:654] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:04.329059 11377 tablet_bootstrap.cc:492] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3: Bootstrap starting.
I20260809 08:43:04.329433 11410 ts_tablet_manager.cc:1918] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:43:04.329464 11377 tablet_bootstrap.cc:654] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:04.329480 11375 tablet_bootstrap.cc:492] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007: Bootstrap starting.
I20260809 08:43:04.329493 11410 log.cc:1196] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/55696b0cf71a4c23a1829b7b377878fe
I20260809 08:43:04.329847 11375 tablet_bootstrap.cc:654] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:04.329916 11410 ts_tablet_manager.cc:1939] T 55696b0cf71a4c23a1829b7b377878fe P b29a7e8c2c414b968de49a8d00114007: Deleting consensus metadata
I20260809 08:43:04.330224 10630 catalog_manager.cc:4841] TS b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429): tablet 55696b0cf71a4c23a1829b7b377878fe (table test-table [id=9b3d98f4904c484893c511b6a443f889]) successfully deleted
I20260809 08:43:04.330348 11375 tablet_bootstrap.cc:492] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007: No bootstrap required, opened a new log
I20260809 08:43:04.330390 11375 ts_tablet_manager.cc:1397] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:04.330497 11375 raft_consensus.cc:348] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.330576 11375 raft_consensus.cc:374] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:04.330600 11375 raft_consensus.cc:729] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29a7e8c2c414b968de49a8d00114007, State: Initialized, Role: FOLLOWER
I20260809 08:43:04.330610 11409 ts_tablet_manager.cc:1918] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:43:04.330657 11409 log.cc:1196] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/55696b0cf71a4c23a1829b7b377878fe
I20260809 08:43:04.330655 11375 consensus_queue.cc:260] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.330766 11375 ts_tablet_manager.cc:1428] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:04.330863 11377 tablet_bootstrap.cc:492] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3: No bootstrap required, opened a new log
I20260809 08:43:04.330879 11409 ts_tablet_manager.cc:1939] T 55696b0cf71a4c23a1829b7b377878fe P f7d927b63ca04493b9363616bbc03660: Deleting consensus metadata
I20260809 08:43:04.330907 11377 ts_tablet_manager.cc:1397] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:43:04.331017 11377 raft_consensus.cc:348] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.331079 11377 raft_consensus.cc:374] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:04.331099 11377 raft_consensus.cc:729] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f43232019c4e9e9ff3900c642028a3, State: Initialized, Role: FOLLOWER
I20260809 08:43:04.331141 11377 consensus_queue.cc:260] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.331240 11377 ts_tablet_manager.cc:1428] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:04.331284 10631 catalog_manager.cc:4841] TS f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265): tablet 55696b0cf71a4c23a1829b7b377878fe (table test-table [id=9b3d98f4904c484893c511b6a443f889]) successfully deleted
I20260809 08:43:04.331722 11379 tablet_bootstrap.cc:492] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660: No bootstrap required, opened a new log
I20260809 08:43:04.331795 11379 ts_tablet_manager.cc:1397] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 08:43:04.331936 11379 raft_consensus.cc:348] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.332008 11379 raft_consensus.cc:374] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:04.332041 11379 raft_consensus.cc:729] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7d927b63ca04493b9363616bbc03660, State: Initialized, Role: FOLLOWER
I20260809 08:43:04.332093 11379 consensus_queue.cc:260] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.332182 11379 ts_tablet_manager.cc:1428] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:04.494087 11386 raft_consensus.cc:482] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:43:04.494225 11386 raft_consensus.cc:504] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.494601 11386 leader_election.cc:288] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429), 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063)
I20260809 08:43:04.497830 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.494912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:38243 (local address 127.31.63.66:45429)
0809 08:43:04.494997 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.495000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.495093 (+ 93us) server_negotiation.cc:407] Connection header received
0809 08:43:04.495193 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.495194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.495211 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.495280 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:04.495460 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.495957 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.497041 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.497423 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.497520 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.497553 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.497721 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.497758 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 08:43:04.497851 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.495099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:46573 (local address 127.31.63.67:37063)
0809 08:43:04.495157 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.495159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.495168 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:43:04.495288 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.495289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.495313 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.495376 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:04.495557 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.496022 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.497037 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.497419 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.497510 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.497520 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.497727 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.497772 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:43:04.497884 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.495006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:37063 (local address 127.31.63.65:46573)
0809 08:43:04.495112 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:04.495122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:04.495186 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:04.495397 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:04.495398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:04.495400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:04.495480 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.495482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.496113 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.496115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.496936 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.496939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.497510 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.497511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.497620 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.497624 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:43:04.497734 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 08:43:04.497780 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":3,"thread_start_us":40,"threads_started":1}
I20260809 08:43:04.497859 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.494821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:45429 (local address 127.31.63.65:38243)
0809 08:43:04.494982 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:04.495013 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:04.495121 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:04.495299 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:04.495300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:04.495301 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:04.495385 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.495387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.496059 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.496060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.496940 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.496942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.497563 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.497565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.497636 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.497640 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:43:04.497712 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:43:04.497752 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":43,"thread_start_us":62,"threads_started":1}
I20260809 08:43:04.498159 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630ff5ab1d7477781a5812940c70a6a" candidate_uuid: "f7d927b63ca04493b9363616bbc03660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f43232019c4e9e9ff3900c642028a3" is_pre_election: true
I20260809 08:43:04.498268 11265 raft_consensus.cc:2454] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7d927b63ca04493b9363616bbc03660 in term 0.
I20260809 08:43:04.498355 11057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630ff5ab1d7477781a5812940c70a6a" candidate_uuid: "f7d927b63ca04493b9363616bbc03660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b29a7e8c2c414b968de49a8d00114007" is_pre_election: true
I20260809 08:43:04.498449 11057 raft_consensus.cc:2454] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7d927b63ca04493b9363616bbc03660 in term 0.
I20260809 08:43:04.498473 10772 leader_election.cc:302] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f43232019c4e9e9ff3900c642028a3, f7d927b63ca04493b9363616bbc03660; no voters:
I20260809 08:43:04.498625 11386 raft_consensus.cc:2781] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:43:04.498720 11386 raft_consensus.cc:482] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:43:04.498755 11386 raft_consensus.cc:3037] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:04.499500 11386 raft_consensus.cc:504] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.499644 11386 leader_election.cc:288] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [CANDIDATE]: Term 1 election: Requested vote from peers b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429), 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063)
I20260809 08:43:04.499853 11057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630ff5ab1d7477781a5812940c70a6a" candidate_uuid: "f7d927b63ca04493b9363616bbc03660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b29a7e8c2c414b968de49a8d00114007"
I20260809 08:43:04.499879 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4630ff5ab1d7477781a5812940c70a6a" candidate_uuid: "f7d927b63ca04493b9363616bbc03660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f43232019c4e9e9ff3900c642028a3"
I20260809 08:43:04.499933 11057 raft_consensus.cc:3037] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:04.499939 11265 raft_consensus.cc:3037] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:04.500576 11057 raft_consensus.cc:2454] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7d927b63ca04493b9363616bbc03660 in term 1.
I20260809 08:43:04.500582 11265 raft_consensus.cc:2454] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7d927b63ca04493b9363616bbc03660 in term 1.
I20260809 08:43:04.500801 10772 leader_election.cc:302] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b29a7e8c2c414b968de49a8d00114007, f7d927b63ca04493b9363616bbc03660; no voters:
I20260809 08:43:04.500891 11386 raft_consensus.cc:2781] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:04.501048 11386 raft_consensus.cc:686] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [term 1 LEADER]: Becoming Leader. State: Replica: f7d927b63ca04493b9363616bbc03660, State: Running, Role: LEADER
I20260809 08:43:04.501107 11386 consensus_queue.cc:237] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } }
I20260809 08:43:04.501996 10648 catalog_manager.cc:5495] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7d927b63ca04493b9363616bbc03660 (127.31.63.65). New cstate: current_term: 1 leader_uuid: "f7d927b63ca04493b9363616bbc03660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d927b63ca04493b9363616bbc03660" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 } health_report { overall_health: UNKNOWN } } }
I20260809 08:43:04.515843 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.513319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51190 (local address 127.31.63.65:35265)
0809 08:43:04.513358 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.513360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.513368 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:43:04.513452 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.513453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.513478 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.513647 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:43:04.513776 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.514269 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.515031 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.515243 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.515323 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.515491 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.515608 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.515689 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.515728 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.515751 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:43:04.520766 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.518550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:51629 (local address 127.31.63.67:37063)
0809 08:43:04.518607 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.518609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.518748 (+ 139us) server_negotiation.cc:407] Connection header received
0809 08:43:04.518779 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.518781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.518790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.518837 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:43:04.518938 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.519442 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.520150 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.520339 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.520387 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.520569 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.520596 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.520634 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.520693 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.520711 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:43:04.520790 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.518461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:37063 (local address 127.31.63.65:51629)
0809 08:43:04.518679 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:04.518688 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:04.518742 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:04.518837 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:04.518838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:04.518839 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:43:04.518887 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.518888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.519519 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.519520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.520079 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.520081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.520403 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.520404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.520453 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.520471 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.520650 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.520651 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:43:04.520683 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 08:43:04.520720 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":109,"threads_started":1}
I20260809 08:43:04.523131 11057 raft_consensus.cc:1261] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Refusing update from remote peer f7d927b63ca04493b9363616bbc03660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:04.523139 11265 raft_consensus.cc:1261] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Refusing update from remote peer f7d927b63ca04493b9363616bbc03660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:04.523422 11490 consensus_queue.cc:1035] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b29a7e8c2c414b968de49a8d00114007" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:04.523545 11386 consensus_queue.cc:1035] T 4630ff5ab1d7477781a5812940c70a6a P f7d927b63ca04493b9363616bbc03660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41f43232019c4e9e9ff3900c642028a3" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 37063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:04.531490 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.528967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35265 (local address 127.31.63.67:39981)
0809 08:43:04.529031 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:04.529043 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:04.529082 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:04.529393 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:04.529394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:04.529395 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:43:04.529437 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.529438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.530070 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.530072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.530629 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:04.530630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.531057 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.531059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:04.531128 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.531136 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.531336 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.531338 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:43:04.531384 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 08:43:04.531441 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 08:43:04.531488 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.529209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:39981 (local address 127.31.63.65:35265)
0809 08:43:04.529268 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:04.529269 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:04.529277 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:43:04.529302 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:04.529303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:04.529312 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:04.529389 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:43:04.529487 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.529986 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.530726 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:04.530981 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:04.531038 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:04.531221 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:43:04.531257 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:43:04.531307 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:04.531387 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 08:43:04.531413 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:43:04.538206 10648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:43:04.538314 10648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:43:04.538935 10648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2ba1c415c2841f481b7e7feb9ac4d9e: Sending DeleteTablet for 3 replicas of tablet 4630ff5ab1d7477781a5812940c70a6a
I20260809 08:43:04.539213 11034 tablet_service.cc:1514] Processing DeleteTablet for tablet 4630ff5ab1d7477781a5812940c70a6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:04 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:43872
I20260809 08:43:04.539228 11237 tablet_service.cc:1514] Processing DeleteTablet for tablet 4630ff5ab1d7477781a5812940c70a6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:04 UTC) from {username='oryx', principal='oryx/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:48758
I20260809 08:43:04.539355 11410 tablet_replica.cc:330] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007: stopping tablet replica
I20260809 08:43:04.539395 11411 tablet_replica.cc:330] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3: stopping tablet replica
I20260809 08:43:04.539424 11410 raft_consensus.cc:2229] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:43:04.539448 11411 raft_consensus.cc:2229] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:43:04.539458 11410 raft_consensus.cc:2258] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:04.539489 11411 raft_consensus.cc:2258] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:04.539795 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 10748
I20260809 08:43:04.539963 11410 ts_tablet_manager.cc:1905] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:04.542340 11410 ts_tablet_manager.cc:1918] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:43:04.542393 11410 log.cc:1196] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/4630ff5ab1d7477781a5812940c70a6a
I20260809 08:43:04.542613 11410 ts_tablet_manager.cc:1939] T 4630ff5ab1d7477781a5812940c70a6a P b29a7e8c2c414b968de49a8d00114007: Deleting consensus metadata
I20260809 08:43:04.542906 10630 catalog_manager.cc:4841] TS b29a7e8c2c414b968de49a8d00114007 (127.31.63.66:45429): tablet 4630ff5ab1d7477781a5812940c70a6a (table test-table [id=7d5bd2b6362b4fed9093e6626b695af4]) successfully deleted
I20260809 08:43:04.551641 11411 ts_tablet_manager.cc:1905] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:04.554816 11411 ts_tablet_manager.cc:1918] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:43:04.554909 11411 log.cc:1196] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/4630ff5ab1d7477781a5812940c70a6a
I20260809 08:43:04.555169 11411 ts_tablet_manager.cc:1939] T 4630ff5ab1d7477781a5812940c70a6a P 41f43232019c4e9e9ff3900c642028a3: Deleting consensus metadata
I20260809 08:43:04.556226 10630 catalog_manager.cc:4841] TS 41f43232019c4e9e9ff3900c642028a3 (127.31.63.67:37063): tablet 4630ff5ab1d7477781a5812940c70a6a (table test-table [id=7d5bd2b6362b4fed9093e6626b695af4]) successfully deleted
W20260809 08:43:04.561007 10631 connection.cc:535] client connection to 127.31.63.65:35265 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 08:43:04.561177 10631 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 08:43:04.561425 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 10946
I20260809 08:43:04.566998 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:04.561589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35265 (local address 127.0.0.1:51196)
0809 08:43:04.566901 (+ 5312us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:04.566960 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:35265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5290,"thread_start_us":44,"threads_started":1}
W20260809 08:43:04.567157 10631 catalog_manager.cc:4584] TS f7d927b63ca04493b9363616bbc03660 (127.31.63.65:35265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4630ff5ab1d7477781a5812940c70a6a: Network error: Client connection negotiation failed: client connection to 127.31.63.65:35265: connect: Connection refused (error 111)
I20260809 08:43:04.574209 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 11133
I20260809 08:43:04.587921 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 10602
2026-08-09T08:43:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3111 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:43:04 dist-test-slave-gkw7 krb5kdc[11517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:43:04 dist-test-slave-gkw7 krb5kdc[11517](info): set up 2 sockets
Aug 09 08:43:04 dist-test-slave-gkw7 krb5kdc[11517](info): commencing operation
krb5kdc: starting...
W20260809 08:43:05.828850 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:43:05 dist-test-slave-gkw7 krb5kdc[11517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:43:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:43:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:43:05.956322 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:34063
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:34063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:06.039429 11532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:06.039642 11532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:06.039666 11532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:06.040740 11532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:43:06.040779 11532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:06.040794 11532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:06.040807 11532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:43:06.040818 11532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:43:06.042268 11532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:34063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:34063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:06.042465 11532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:06.042645 11532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:06.047696 11540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:06.047701 11543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:06.047703 11541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:06.048097 11532 server_base.cc:1034] running on GCE node
I20260809 08:43:06.048390 11532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:06.048687 11532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:06.049829 11532 hybrid_clock.cc:648] HybridClock initialized: now 1786264986049809 us; error 34 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T08:43:06Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1460 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:43:06 dist-test-slave-gkw7 krb5kdc[11548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:43:06 dist-test-slave-gkw7 krb5kdc[11548](info): set up 2 sockets
Aug 09 08:43:06 dist-test-slave-gkw7 krb5kdc[11548](info): commencing operation
krb5kdc: starting...
W20260809 08:43:07.284108 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:43:07 dist-test-slave-gkw7 krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:43:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:43:07.413446 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:42219
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:42219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:07.501188 11565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:07.501412 11565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:07.501435 11565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:07.502658 11565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:43:07.502717 11565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:07.502739 11565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:07.502761 11565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:43:07.502792 11565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:43:07.504273 11565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:42219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:42219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:07.504474 11565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:07.504642 11565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:07.509676 11571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:07.509685 11573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:07.509685 11576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:07.509912 11565 server_base.cc:1034] running on GCE node
I20260809 08:43:07.510210 11565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:07.510496 11565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:07.511664 11565 hybrid_clock.cc:648] HybridClock initialized: now 1786264987511644 us; error 39 us; skew 500 ppm
Aug 09 08:43:07 dist-test-slave-gkw7 krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264987, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:07.520951 11565 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:43:07.521487 11565 webserver.cc:460] Webserver started at http://127.31.63.126:41847/ using document root <none> and password file <none>
I20260809 08:43:07.521772 11565 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:07.521858 11565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:07.522017 11565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:07.523104 11565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "f06def494cec4ce49ab7a5cda487accb"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "82036fefcb135e8cef63f495839c5270"
server_key_iv: "09410d536f95343c992ead3b69a9a40f"
server_key_version: "encryptionkey@0"
I20260809 08:43:07.523486 11565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "f06def494cec4ce49ab7a5cda487accb"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "82036fefcb135e8cef63f495839c5270"
server_key_iv: "09410d536f95343c992ead3b69a9a40f"
server_key_version: "encryptionkey@0"
I20260809 08:43:07.524890 11565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:43:07.525759 11585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:07.525955 11565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:43:07.526070 11565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "f06def494cec4ce49ab7a5cda487accb"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "82036fefcb135e8cef63f495839c5270"
server_key_iv: "09410d536f95343c992ead3b69a9a40f"
server_key_version: "encryptionkey@0"
I20260809 08:43:07.526176 11565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:07.539989 11565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:07.542373 11565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:07.542548 11565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:07.547008 11565 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:42219
I20260809 08:43:07.547061 11658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:42219 every 8 connection(s)
I20260809 08:43:07.547469 11565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:43:07.548426 11659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:07.549887 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 11565
I20260809 08:43:07.550014 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:43:07.550233 31997 external_mini_cluster.cc:1428] Setting key a82945c5e13974a6c549debfa9b6785a
I20260809 08:43:07.551403 11659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb: Bootstrap starting.
I20260809 08:43:07.552186 11659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:07.552572 11659 log.cc:824] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:07.553294 11659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb: No bootstrap required, opened a new log
I20260809 08:43:07.555724 11659 raft_consensus.cc:348] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } }
I20260809 08:43:07.555886 11659 raft_consensus.cc:374] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:07.555927 11659 raft_consensus.cc:729] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06def494cec4ce49ab7a5cda487accb, State: Initialized, Role: FOLLOWER
I20260809 08:43:07.556057 11659 consensus_queue.cc:260] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } }
I20260809 08:43:07.556159 11659 raft_consensus.cc:388] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:43:07.556197 11659 raft_consensus.cc:482] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:43:07.556241 11659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:07.557039 11659 raft_consensus.cc:504] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } }
I20260809 08:43:07.557171 11659 leader_election.cc:302] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f06def494cec4ce49ab7a5cda487accb; no voters:
I20260809 08:43:07.557385 11659 leader_election.cc:288] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:43:07.557412 11667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:07.557554 11667 raft_consensus.cc:686] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 1 LEADER]: Becoming Leader. State: Replica: f06def494cec4ce49ab7a5cda487accb, State: Running, Role: LEADER
I20260809 08:43:07.557703 11659 sys_catalog.cc:564] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:43:07.557767 11667 consensus_queue.cc:237] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } }
Aug 09 08:43:07 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:43:07.558173 11668 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f06def494cec4ce49ab7a5cda487accb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } } }
I20260809 08:43:07.558360 11668 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:07.558202 11669 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f06def494cec4ce49ab7a5cda487accb. Latest consensus state: current_term: 1 leader_uuid: "f06def494cec4ce49ab7a5cda487accb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } } }
I20260809 08:43:07.558683 11669 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:07.558682 11680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:43:07.559788 11680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:43:07.561765 11680 catalog_manager.cc:1347] Generated new cluster ID: 0a246ddc361645caab7916e4709a58ad
I20260809 08:43:07.561838 11680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:43:07.562737 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:07.550816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.31.63.126:42219)
0809 08:43:07.550999 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:07.551007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:07.551034 (+ 27us) server_negotiation.cc:407] Connection header received
0809 08:43:07.551472 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:07.551481 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:07.551552 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:07.551670 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:07.552537 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:07.553716 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:07.554808 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:07.555271 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:07.555407 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:07.558619 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:07.558624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:07.558624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:07.558639 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:07.560860 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:07.561319 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:07.561323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:07.561323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:07.561438 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:07.561703 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:07.561704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:07.561704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:07.562057 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:07.562192 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:07.562360 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 08:43:07.562508 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260809 08:43:07.584970 11680 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:43:07.585783 11680 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:43:07.591197 11680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb: Generated new TSK 0
I20260809 08:43:07.591447 11680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:43:07.628157 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42219
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:07.715786 11701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:07.716028 11701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:07.716071 11701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:07.717267 11701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:07.717337 11701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:07.717382 11701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:43:07.718942 11701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:07.719233 11701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:07.719480 11701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:07.724659 11712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:07.724681 11707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:07.724694 11709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:07.725100 11701 server_base.cc:1034] running on GCE node
I20260809 08:43:07.725322 11701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:07.725521 11701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:07.726749 11701 hybrid_clock.cc:648] HybridClock initialized: now 1786264987726728 us; error 38 us; skew 500 ppm
Aug 09 08:43:07 dist-test-slave-gkw7 krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264987, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:07.729439 11701 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:43:07.729970 11701 webserver.cc:460] Webserver started at http://127.31.63.65:46339/ using document root <none> and password file <none>
I20260809 08:43:07.730195 11701 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:07.730240 11701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:07.730334 11701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:07.731454 11701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "9d8aa45e5a6b4b98a0e35a6a2625032b"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "ff66c1c32f12a61804a20db6fed35561"
server_key_iv: "0b227b1a70f7ef0d71ce07666326d0fc"
server_key_version: "encryptionkey@0"
I20260809 08:43:07.731773 11701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "9d8aa45e5a6b4b98a0e35a6a2625032b"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "ff66c1c32f12a61804a20db6fed35561"
server_key_iv: "0b227b1a70f7ef0d71ce07666326d0fc"
server_key_version: "encryptionkey@0"
I20260809 08:43:07.733361 11701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:43:07.734213 11723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:07.734378 11701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:43:07.734472 11701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "9d8aa45e5a6b4b98a0e35a6a2625032b"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "ff66c1c32f12a61804a20db6fed35561"
server_key_iv: "0b227b1a70f7ef0d71ce07666326d0fc"
server_key_version: "encryptionkey@0"
I20260809 08:43:07.734529 11701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:07.750488 11701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:07.752403 11701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:07.752518 11701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:07.752722 11701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:07.753052 11701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:07.753086 11701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:07.753109 11701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:07.753155 11701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:07.759461 11701 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:43887
I20260809 08:43:07.759526 11892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:43887 every 8 connection(s)
I20260809 08:43:07.759881 11701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:43:07.764477 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 11701
I20260809 08:43:07.764599 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:43:07.765185 31997 external_mini_cluster.cc:1428] Setting key d54cebe905388c322e88279cd4f97f4b
Aug 09 08:43:07 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264987, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:43:07.769871 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:07.760660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:56975 (local address 127.31.63.126:42219)
0809 08:43:07.760731 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:07.760733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:07.760944 (+ 211us) server_negotiation.cc:407] Connection header received
0809 08:43:07.761437 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:07.761438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:07.761449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:07.761505 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:07.762534 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:07.763038 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:07.764022 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:07.764218 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:07.764279 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:07.766773 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:07.766774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:07.766775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:07.766785 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:07.768314 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:07.768829 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:07.768830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:07.768830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:07.768884 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:07.769181 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:07.769182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:07.769182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:07.769301 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:07.769353 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:07.769724 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 08:43:07.769804 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:43:07.770170 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:07.760618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:56975)
0809 08:43:07.760817 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:07.760861 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:07.761354 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:07.761633 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:07.761636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:07.762235 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:07.762449 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:07.762455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:07.763122 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:07.763123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:07.763946 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:07.763948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:07.764318 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:07.764319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:07.764483 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:07.765083 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:07.765094 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:07.766684 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:07.768403 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:07.768411 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:07.768413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:07.768745 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:07.768953 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:07.768954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:07.768955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:07.769113 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:07.769371 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:07.769379 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:07.769579 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 08:43:07.769700 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 08:43:07.769892 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260809 08:43:07.770784 11895 heartbeater.cc:347] Connected to a master server at 127.31.63.126:42219
I20260809 08:43:07.770897 11895 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:07.771126 11895 heartbeater.cc:511] Master 127.31.63.126:42219 requested a full tablet report, sending...
I20260809 08:43:07.771677 11606 ts_manager.cc:194] Registered new tserver with Master: 9d8aa45e5a6b4b98a0e35a6a2625032b (127.31.63.65:43887)
I20260809 08:43:07.772315 11606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:56975
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:43:07.792634 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42219
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:07.881253 11902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:07.881475 11902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:07.881501 11902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:07.882797 11902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:07.882846 11902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:07.882874 11902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:43:07.884387 11902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:07.884662 11902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:07.884912 11902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:07.890375 11914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:07.890404 11911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:07.890522 11902 server_base.cc:1034] running on GCE node
W20260809 08:43:07.890625 11912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:07.890866 11902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:07.891075 11902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:07.892218 11902 hybrid_clock.cc:648] HybridClock initialized: now 1786264987892202 us; error 41 us; skew 500 ppm
Aug 09 08:43:07 dist-test-slave-gkw7 krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264987, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:07.894896 11902 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:43:07.895402 11902 webserver.cc:460] Webserver started at http://127.31.63.66:39065/ using document root <none> and password file <none>
I20260809 08:43:07.895663 11902 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:07.895711 11902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:07.895888 11902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:07.896999 11902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "35f97af497e043f6954416bcb35fc9fa"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "29a6cfddc2bcd8bc0ecc9d69e9f660a4"
server_key_iv: "51f30bef6e20fd138803ee959ef61929"
server_key_version: "encryptionkey@0"
I20260809 08:43:07.897382 11902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "35f97af497e043f6954416bcb35fc9fa"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "29a6cfddc2bcd8bc0ecc9d69e9f660a4"
server_key_iv: "51f30bef6e20fd138803ee959ef61929"
server_key_version: "encryptionkey@0"
I20260809 08:43:07.898856 11902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:43:07.899720 11923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:07.899971 11902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:07.900070 11902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "35f97af497e043f6954416bcb35fc9fa"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "29a6cfddc2bcd8bc0ecc9d69e9f660a4"
server_key_iv: "51f30bef6e20fd138803ee959ef61929"
server_key_version: "encryptionkey@0"
I20260809 08:43:07.900149 11902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:07.923677 11902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:07.926301 11902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:07.926466 11902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:07.926746 11902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:07.927100 11902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:07.927160 11902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:07.927206 11902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:07.927263 11902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:07.934343 11902 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:36383
I20260809 08:43:07.934386 12096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:36383 every 8 connection(s)
I20260809 08:43:07.934746 11902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:43:07.938835 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 11902
I20260809 08:43:07.938956 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:43:07.939209 31997 external_mini_cluster.cc:1428] Setting key 038ce5f7e896f29624e6b743c3dc4a8e
Aug 09 08:43:07 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264987, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:43:07.946682 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:07.935531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:36539 (local address 127.31.63.126:42219)
0809 08:43:07.935588 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:07.935591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:07.935809 (+ 218us) server_negotiation.cc:407] Connection header received
0809 08:43:07.936472 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:07.936474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:07.936490 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:07.936577 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:07.937845 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:07.938489 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:07.940120 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:07.940323 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:07.940386 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:07.943444 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:07.943446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:07.943446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:07.943453 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:07.944883 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:07.945479 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:07.945480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:07.945480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:07.945592 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:07.945949 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:07.945950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:07.945950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:07.946068 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:07.946124 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:07.946538 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 08:43:07.946611 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:43:07.947064 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:07.935488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:36539)
0809 08:43:07.935666 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:07.935713 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:07.936321 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:07.936698 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:07.936702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:07.937476 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:07.937727 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:07.937734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:07.938597 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:07.938599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:07.940024 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:07.940028 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:07.940423 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:07.940424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:07.940594 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:07.941184 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:07.941196 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:07.943302 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:07.944979 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:07.944988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:07.944991 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:07.945396 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:07.945671 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:07.945672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:07.945672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:07.945869 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:07.946152 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:07.946160 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:07.946368 (+ 208us) client_negotiation.cc:769] Sending connection context
0809 08:43:07.946511 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 08:43:07.946732 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260809 08:43:07.947680 12098 heartbeater.cc:347] Connected to a master server at 127.31.63.126:42219
I20260809 08:43:07.947803 12098 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:07.948042 12098 heartbeater.cc:511] Master 127.31.63.126:42219 requested a full tablet report, sending...
I20260809 08:43:07.948520 11606 ts_manager.cc:194] Registered new tserver with Master: 35f97af497e043f6954416bcb35fc9fa (127.31.63.66:36383)
I20260809 08:43:07.948916 11606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:36539
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:43:07.972641 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42219
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:08.065618 12108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:08.065855 12108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:08.065882 12108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:08.067130 12108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:08.067179 12108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:08.067207 12108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:43:08.068805 12108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:42219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:08.069096 12108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:08.069365 12108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:43:08.075126 12108 server_base.cc:1034] running on GCE node
W20260809 08:43:08.075083 12119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:08.075096 12116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:08.075095 12115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:08.075531 12108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:08.075809 12108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:08.076956 12108 hybrid_clock.cc:648] HybridClock initialized: now 1786264988076938 us; error 37 us; skew 500 ppm
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264988, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:08.079658 12108 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:43:08.080070 12108 webserver.cc:460] Webserver started at http://127.31.63.67:41181/ using document root <none> and password file <none>
I20260809 08:43:08.080284 12108 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:08.080327 12108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:08.080420 12108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:08.081466 12108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "73410d4e1d784a9dbd10773c9c5afa49"
format_stamp: "Formatted at 2026-08-09 08:43:08 on dist-test-slave-gkw7"
server_key: "38c61f80e694cfdeaf541cd81cf00ef6"
server_key_iv: "9d3185916296c73c763d604cd331d328"
server_key_version: "encryptionkey@0"
I20260809 08:43:08.081887 12108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "73410d4e1d784a9dbd10773c9c5afa49"
format_stamp: "Formatted at 2026-08-09 08:43:08 on dist-test-slave-gkw7"
server_key: "38c61f80e694cfdeaf541cd81cf00ef6"
server_key_iv: "9d3185916296c73c763d604cd331d328"
server_key_version: "encryptionkey@0"
I20260809 08:43:08.083276 12108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:08.083988 12130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.084216 12108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:08.084298 12108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "73410d4e1d784a9dbd10773c9c5afa49"
format_stamp: "Formatted at 2026-08-09 08:43:08 on dist-test-slave-gkw7"
server_key: "38c61f80e694cfdeaf541cd81cf00ef6"
server_key_iv: "9d3185916296c73c763d604cd331d328"
server_key_version: "encryptionkey@0"
I20260809 08:43:08.084390 12108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:08.103972 12108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:08.106351 12108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:08.106529 12108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:08.106802 12108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:08.107157 12108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:08.107193 12108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.107216 12108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:08.107264 12108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.113674 12108 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:33001
I20260809 08:43:08.113758 12296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:33001 every 8 connection(s)
I20260809 08:43:08.114132 12108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:43:08.119274 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 12108
I20260809 08:43:08.119395 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:43:08.119578 31997 external_mini_cluster.cc:1428] Setting key 12ec35aaccbee5f4857e36f236da24dc
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264988, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:43:08.124118 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.114953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:60379 (local address 127.31.63.126:42219)
0809 08:43:08.115025 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.115027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.115204 (+ 177us) server_negotiation.cc:407] Connection header received
0809 08:43:08.115717 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.115719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.115730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.115801 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.116809 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.117312 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.118389 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.118576 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.118632 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.121290 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:08.121292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:08.121292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:08.121299 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:08.122669 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:08.123087 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:08.123088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:08.123089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:08.123171 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:08.123410 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:08.123411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:08.123411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:08.123524 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:08.123575 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:08.123937 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 08:43:08.124030 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:43:08.124191 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.114861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:60379)
0809 08:43:08.115048 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:08.115092 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:08.115622 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:08.115867 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:08.115870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:08.116466 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:08.116710 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.116716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.117401 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.117402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.118311 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.118314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.118663 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.118664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.118783 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.119413 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:08.119428 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:08.121195 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:08.122760 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:08.122768 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:08.122770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:08.123009 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:08.123233 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:08.123234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:08.123234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:08.123352 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:08.123585 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:08.123590 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:08.123737 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 08:43:08.123828 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 08:43:08.123965 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260809 08:43:08.124747 12298 heartbeater.cc:347] Connected to a master server at 127.31.63.126:42219
I20260809 08:43:08.124861 12298 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:08.125140 12298 heartbeater.cc:511] Master 127.31.63.126:42219 requested a full tablet report, sending...
I20260809 08:43:08.125710 11606 ts_manager.cc:194] Registered new tserver with Master: 73410d4e1d784a9dbd10773c9c5afa49 (127.31.63.67:33001)
I20260809 08:43:08.126102 11606 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:60379
I20260809 08:43:08.131732 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab.
Entry for principal oryx/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab.
WARNING: no policy specified for oryx/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab.
Entry for principal oryx/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab.
WARNING: no policy specified for oryx/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab.
Entry for principal oryx/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab.
I20260809 08:43:08.216670 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 11701
I20260809 08:43:08.231043 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 11902
I20260809 08:43:08.247429 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 12108
I20260809 08:43:08.263168 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 11565
I20260809 08:43:08.275050 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:42219
--webserver_interface=127.31.63.126
--webserver_port=41847
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:42219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:08.358772 12316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:08.358980 12316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:08.359002 12316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:08.360154 12316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:43:08.360205 12316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:08.360219 12316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:08.360232 12316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:43:08.360244 12316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:43:08.362110 12316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:42219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:42219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=41847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:08.362398 12316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:08.362646 12316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:08.367563 12323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:08.367719 12316 server_base.cc:1034] running on GCE node
W20260809 08:43:08.367570 12321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:08.367614 12326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:08.368074 12316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:08.368410 12316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:08.369589 12316 hybrid_clock.cc:648] HybridClock initialized: now 1786264988369567 us; error 40 us; skew 500 ppm
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264988, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:08.372043 12316 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:43:08.372542 12316 webserver.cc:460] Webserver started at http://127.31.63.126:41847/ using document root <none> and password file <none>
I20260809 08:43:08.372804 12316 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:08.372896 12316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:08.374392 12316 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:08.375090 12336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.375272 12316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:08.375366 12316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "f06def494cec4ce49ab7a5cda487accb"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "82036fefcb135e8cef63f495839c5270"
server_key_iv: "09410d536f95343c992ead3b69a9a40f"
server_key_version: "encryptionkey@0"
I20260809 08:43:08.375741 12316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:08.431731 12316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:08.434085 12316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:08.434257 12316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:08.438680 12316 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:42219
I20260809 08:43:08.438730 12418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:42219 every 8 connection(s)
I20260809 08:43:08.439163 12316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:43:08.440747 12419 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:43:08.441205 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 12316
I20260809 08:43:08.441504 12419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb: Bootstrap starting.
I20260809 08:43:08.441516 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:43887
--local_ip_for_outbound_sockets=127.31.63.65
--tserver_master_addrs=127.31.63.126:42219
--webserver_port=46339
--webserver_interface=127.31.63.65
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.63.65
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:43:08.451118 12419 log.cc:824] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:08.452996 12419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:43:08.453337 12419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb: Bootstrap complete.
I20260809 08:43:08.455829 12419 raft_consensus.cc:348] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } }
I20260809 08:43:08.456100 12419 raft_consensus.cc:729] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06def494cec4ce49ab7a5cda487accb, State: Initialized, Role: FOLLOWER
I20260809 08:43:08.456308 12419 consensus_queue.cc:260] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } }
I20260809 08:43:08.456398 12419 raft_consensus.cc:388] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:43:08.456455 12419 raft_consensus.cc:482] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:43:08.456509 12419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:43:08.457630 12419 raft_consensus.cc:504] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } }
I20260809 08:43:08.457756 12419 leader_election.cc:302] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f06def494cec4ce49ab7a5cda487accb; no voters:
I20260809 08:43:08.457964 12419 leader_election.cc:288] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:43:08.458036 12425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:43:08.458210 12425 raft_consensus.cc:686] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [term 2 LEADER]: Becoming Leader. State: Replica: f06def494cec4ce49ab7a5cda487accb, State: Running, Role: LEADER
I20260809 08:43:08.458268 12419 sys_catalog.cc:564] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:43:08.458400 12425 consensus_queue.cc:237] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } }
I20260809 08:43:08.458688 12425 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f06def494cec4ce49ab7a5cda487accb. Latest consensus state: current_term: 2 leader_uuid: "f06def494cec4ce49ab7a5cda487accb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } } }
I20260809 08:43:08.458860 12425 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:08.458689 12426 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f06def494cec4ce49ab7a5cda487accb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06def494cec4ce49ab7a5cda487accb" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 42219 } } }
I20260809 08:43:08.459209 12426 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:08.459225 12436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:43:08.459802 12436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:43:08.459987 12436 catalog_manager.cc:1259] Loaded cluster ID: 0a246ddc361645caab7916e4709a58ad
I20260809 08:43:08.460158 12436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:43:08.461409 12436 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:43:08.461612 12436 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f06def494cec4ce49ab7a5cda487accb: Loaded TSK: 0
I20260809 08:43:08.461851 12436 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:43:08.529662 12421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:08.529879 12421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:08.529904 12421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:08.531071 12421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:08.531117 12421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:08.531143 12421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:43:08.532589 12421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.65.keytab
--principal=oryx/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:43887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=46339
--tserver_master_addrs=127.31.63.126:42219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:08.532845 12421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:08.533095 12421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:08.539567 12453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:08.539610 12456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:08.539616 12454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:08.539937 12421 server_base.cc:1034] running on GCE node
I20260809 08:43:08.540220 12421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:08.540498 12421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:08.541644 12421 hybrid_clock.cc:648] HybridClock initialized: now 1786264988541623 us; error 37 us; skew 500 ppm
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264988, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:08.544265 12421 init.cc:377] Logged in from keytab as oryx/127.31.63.65@KRBTEST.COM (short username oryx)
I20260809 08:43:08.544775 12421 webserver.cc:460] Webserver started at http://127.31.63.65:46339/ using document root <none> and password file <none>
I20260809 08:43:08.545039 12421 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:08.545125 12421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:08.546607 12421 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:08.547326 12467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.547519 12421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:08.547621 12421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "9d8aa45e5a6b4b98a0e35a6a2625032b"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "ff66c1c32f12a61804a20db6fed35561"
server_key_iv: "0b227b1a70f7ef0d71ce07666326d0fc"
server_key_version: "encryptionkey@0"
I20260809 08:43:08.548007 12421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:08.574254 12421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:08.576517 12421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:08.576699 12421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:08.576972 12421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:08.577345 12421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:08.577404 12421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.577447 12421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:08.577474 12421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.585505 12421 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:43887
I20260809 08:43:08.585628 12629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:43887 every 8 connection(s)
I20260809 08:43:08.586087 12421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:43:08.587864 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 12421
I20260809 08:43:08.588196 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:36383
--local_ip_for_outbound_sockets=127.31.63.66
--tserver_master_addrs=127.31.63.126:42219
--webserver_port=39065
--webserver_interface=127.31.63.66
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.63.66
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:08.596421 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.586820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:39841)
0809 08:43:08.587065 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:08.587115 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:08.587750 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:08.588296 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:08.588301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:08.589084 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:08.589371 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.589379 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.591167 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.591168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.591996 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.591999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.592921 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.592922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.593033 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.593668 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:08.593677 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:08.596271 (+ 2594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":124,"threads_started":1}
W20260809 08:43:08.596659 12639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:08.586955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:39841 (local address 127.31.63.126:42219)
0809 08:43:08.587152 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.587162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.587201 (+ 39us) server_negotiation.cc:407] Connection header received
0809 08:43:08.587996 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.588004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.588072 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.588229 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.589525 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.591008 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.592096 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.592806 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.592931 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.596463 (+ 3532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:39841: BlockingRecv error: recv got EOF from 127.31.63.65:39841 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
W20260809 08:43:08.596676 12631 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:42219 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.63.126:42219: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:08.601933 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.596906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:44477)
0809 08:43:08.596963 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:08.596976 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:08.597018 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:08.597228 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:08.597229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:08.597395 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:08.597458 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.597460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.598133 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.598135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.599062 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.599065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.599413 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.599414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.599501 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.599918 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:08.599924 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:08.601861 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 08:43:08.602178 12639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:08.597012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:44477 (local address 127.31.63.126:42219)
0809 08:43:08.597073 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.597075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.597112 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:43:08.597138 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.597139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.597149 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.597200 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.597574 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.598051 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.599154 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.599335 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.599388 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.602105 (+ 2717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:44477: BlockingRecv error: recv got EOF from 127.31.63.65:44477 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:08.606834 12638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.602330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:44859)
0809 08:43:08.602415 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:08.602425 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:08.602458 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:08.602599 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:08.602600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:08.602737 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:08.602778 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.602779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.603353 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.603354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.604170 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.604171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.604514 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.604515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.604572 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.604956 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:08.604961 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:08.606757 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 08:43:08.606964 12631 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:43:08.607069 12639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:08.602428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:44859 (local address 127.31.63.126:42219)
0809 08:43:08.602481 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.602483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.602498 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:43:08.602516 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.602517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.602525 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.602585 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.602836 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.603284 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.604262 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.604439 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.604495 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.606845 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:44859: BlockingRecv error: recv got EOF from 127.31.63.65:44859 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260809 08:43:08.677027 12640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:08.677295 12640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:08.677338 12640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:08.678682 12640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:08.678761 12640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:08.678804 12640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:43:08.680358 12640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.66.keytab
--principal=oryx/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:36383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=39065
--tserver_master_addrs=127.31.63.126:42219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:08.680639 12640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:08.680882 12640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:08.686312 12647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:08.686594 12640 server_base.cc:1034] running on GCE node
W20260809 08:43:08.686342 12646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:08.686746 12652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:08.687003 12640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:08.687318 12640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:08.688776 12640 hybrid_clock.cc:648] HybridClock initialized: now 1786264988688740 us; error 49 us; skew 500 ppm
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264988, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:08.691500 12640 init.cc:377] Logged in from keytab as oryx/127.31.63.66@KRBTEST.COM (short username oryx)
I20260809 08:43:08.691962 12640 webserver.cc:460] Webserver started at http://127.31.63.66:39065/ using document root <none> and password file <none>
I20260809 08:43:08.692199 12640 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:08.692294 12640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:08.693805 12640 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:08.694545 12661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.694804 12640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:08.694924 12640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "35f97af497e043f6954416bcb35fc9fa"
format_stamp: "Formatted at 2026-08-09 08:43:07 on dist-test-slave-gkw7"
server_key: "29a6cfddc2bcd8bc0ecc9d69e9f660a4"
server_key_iv: "51f30bef6e20fd138803ee959ef61929"
server_key_version: "encryptionkey@0"
I20260809 08:43:08.695326 12640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:08.713428 12640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:08.715483 12640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:08.715660 12640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:08.715922 12640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:08.716264 12640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:08.716321 12640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.716363 12640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:08.716392 12640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.722738 12640 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:36383
I20260809 08:43:08.722784 12830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:36383 every 8 connection(s)
I20260809 08:43:08.723152 12640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:43:08.724839 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 12640
I20260809 08:43:08.725178 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:33001
--local_ip_for_outbound_sockets=127.31.63.67
--tserver_master_addrs=127.31.63.126:42219
--webserver_port=41181
--webserver_interface=127.31.63.67
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:35143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.63.67
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:08.731441 12834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.723884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:37997)
0809 08:43:08.724049 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:08.724098 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:08.724636 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:08.724903 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:08.724909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:08.725851 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:08.726160 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.726166 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.726877 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.726878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.727776 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.727779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.728204 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.728205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.728332 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.728891 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:08.728899 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:08.731297 (+ 2398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
W20260809 08:43:08.731518 12639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:08.723909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:37997 (local address 127.31.63.126:42219)
0809 08:43:08.723977 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.723979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.724184 (+ 205us) server_negotiation.cc:407] Connection header received
0809 08:43:08.724745 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.724746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.724760 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.724823 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.726267 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.726771 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.727878 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.728112 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.728193 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.731427 (+ 3234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:37997: BlockingRecv error: recv got EOF from 127.31.63.66:37997 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 08:43:08.731676 12832 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:42219 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.63.126:42219: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:08.736995 12834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.731929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:47103)
0809 08:43:08.732030 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:08.732040 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:08.732077 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:08.732222 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:08.732222 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:08.732334 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:08.732378 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.732379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.733330 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.733331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.733921 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.733923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.734419 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.734420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.734471 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.734965 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:08.734969 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:08.736951 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 08:43:08.737150 12639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:08.731991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:47103 (local address 127.31.63.126:42219)
0809 08:43:08.732057 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.732060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.732070 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:43:08.732144 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.732146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.732157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.732213 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.732449 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.733222 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.734004 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.734335 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.734416 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.737063 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:47103: BlockingRecv error: recv got EOF from 127.31.63.66:47103 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:08.741496 12834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.737258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:57467)
0809 08:43:08.737406 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:08.737415 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:08.737487 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:08.737638 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:08.737638 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:08.737771 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:08.737797 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.737798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.738415 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.738416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.738901 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.738902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.739248 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.739249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.739302 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.739721 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:08.739725 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:08.741428 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
W20260809 08:43:08.741616 12639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:08.737356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:57467 (local address 127.31.63.126:42219)
0809 08:43:08.737395 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.737397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.737474 (+ 77us) server_negotiation.cc:407] Connection header received
0809 08:43:08.737564 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.737565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.737576 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.737633 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.737882 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.738334 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.738984 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.739174 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.739249 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.741511 (+ 2262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:57467: BlockingRecv error: recv got EOF from 127.31.63.66:57467 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:43:08.741663 12832 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:43:08.814682 12835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:08.814881 12835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:08.814905 12835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:08.816069 12835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:08.816110 12835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:08.816144 12835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:43:08.817641 12835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/krb5kdc/oryx_127.31.63.67.keytab
--principal=oryx/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:33001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=41181
--tserver_master_addrs=127.31.63.126:42219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:08.817817 12835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:08.818065 12835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:43:08.823472 12835 server_base.cc:1034] running on GCE node
W20260809 08:43:08.823426 12843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:08.823455 12847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:08.823449 12842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:08.823877 12835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:08.824124 12835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:08.825297 12835 hybrid_clock.cc:648] HybridClock initialized: now 1786264988825276 us; error 38 us; skew 500 ppm
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786264988, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:08.827809 12835 init.cc:377] Logged in from keytab as oryx/127.31.63.67@KRBTEST.COM (short username oryx)
I20260809 08:43:08.828245 12835 webserver.cc:460] Webserver started at http://127.31.63.67:41181/ using document root <none> and password file <none>
I20260809 08:43:08.828492 12835 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:08.828580 12835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:08.829984 12835 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:43:08.830622 12860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.830811 12835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:43:08.830917 12835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "73410d4e1d784a9dbd10773c9c5afa49"
format_stamp: "Formatted at 2026-08-09 08:43:08 on dist-test-slave-gkw7"
server_key: "38c61f80e694cfdeaf541cd81cf00ef6"
server_key_iv: "9d3185916296c73c763d604cd331d328"
server_key_version: "encryptionkey@0"
I20260809 08:43:08.831274 12835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:08.846589 12835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:08.848601 12835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:08.848776 12835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:08.849031 12835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:08.849370 12835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:08.849426 12835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.849468 12835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:08.849498 12835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:08.855839 12835 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:33001
I20260809 08:43:08.855892 13017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:33001 every 8 connection(s)
I20260809 08:43:08.856246 12835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:43:08.861390 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 12835
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:08.864533 13021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.856956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:51045)
0809 08:43:08.857129 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:08.857200 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:08.857844 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:08.858128 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:08.858132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:08.858813 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:08.859072 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.859077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.859800 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.859801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.860613 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.860615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.861000 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.861001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.861153 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.861812 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:08.861823 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:08.864389 (+ 2566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
W20260809 08:43:08.864566 12639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:08.857004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:51045 (local address 127.31.63.126:42219)
0809 08:43:08.857057 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.857059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.857298 (+ 239us) server_negotiation.cc:407] Connection header received
0809 08:43:08.857985 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.857986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.858000 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.858058 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.859177 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.859697 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.860708 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.860911 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.860991 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.864500 (+ 3509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:51045: BlockingRecv error: recv got EOF from 127.31.63.67:51045 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:43:08.865103 13019 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:42219 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.63.126:42219: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:08.870157 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.861990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.31.63.126:42219)
0809 08:43:08.862132 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.862134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.862235 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:43:08.862286 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.862287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.862299 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.862371 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.863110 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.863748 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.864669 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.865019 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.865087 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.866694 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:08.866698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:08.866698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:08.866708 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:08.868511 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:08.868929 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:08.868933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:08.868933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:08.869063 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:08.869321 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:08.869323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:08.869323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:08.869663 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:08.869822 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:08.869911 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 08:43:08.870039 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260809 08:43:08.870301 13021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.865377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:52767)
0809 08:43:08.865823 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:08.865835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:08.865913 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:08.866053 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:08.866053 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:08.866175 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:08.866220 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.866221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.866996 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.866997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.867493 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.867495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.867861 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.867862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.867934 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.868301 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:08.868305 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:08.870220 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426}
W20260809 08:43:08.870352 12639 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:08.865486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:52767 (local address 127.31.63.126:42219)
0809 08:43:08.865566 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.865567 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.865900 (+ 333us) server_negotiation.cc:407] Connection header received
0809 08:43:08.865970 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.865971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.865982 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.866044 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.866295 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.866913 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.867591 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.867788 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.867861 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.870289 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:52767: BlockingRecv error: recv got EOF from 127.31.63.67:52767 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:08 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:08.874508 13021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:08.870738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:39449)
0809 08:43:08.870781 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:08.870791 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:08.870825 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:08.871008 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:08.871009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:08.871208 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:08.871242 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.871244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.871805 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.871806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.872310 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:08.872312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.872618 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.872618 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:08.872654 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.872975 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:08.872979 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:08.874465 (+ 1486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 08:43:08.874643 13025 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:08.870790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:39449 (local address 127.31.63.126:42219)
0809 08:43:08.870831 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:08.870832 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:08.870842 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:43:08.870930 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:08.870931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:08.870941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:08.870983 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:08.871305 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.871730 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.872376 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:08.872556 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:08.872603 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:08.874571 (+ 1968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:39449: BlockingRecv error: recv got EOF from 127.31.63.67:39449 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:43:08.874640 13019 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 08:43:09 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:09 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:09.612083 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:09.607457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:44679)
0809 08:43:09.607634 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:09.607645 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:09.607699 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:09.607860 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:09.607860 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:09.608033 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:09.608088 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:09.608089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.608752 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.608753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:09.609320 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:09.609322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.609704 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.609705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:09.609752 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:09.610200 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:09.610204 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:09.612011 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":88,"threads_started":1}
W20260809 08:43:09.612233 13028 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:09.607551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:44679 (local address 127.31.63.126:42219)
0809 08:43:09.607720 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:09.607731 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:09.607741 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:43:09.607765 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:09.607765 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:09.607775 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:09.607843 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:09.608163 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.608685 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.609406 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.609628 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.609694 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:09.612140 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:44679: BlockingRecv error: recv got EOF from 127.31.63.65:44679 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
Aug 09 08:43:09 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:09 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:09.746827 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:09.742067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:49769)
0809 08:43:09.742218 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:09.742235 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:09.742303 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:09.742439 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:09.742440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:09.742605 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:09.742659 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:09.742660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.743343 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.743344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:09.743891 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:09.743893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.744232 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.744233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:09.744283 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:09.744845 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:09.744851 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:09.746757 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
W20260809 08:43:09.746963 13028 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:09.742149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:49769 (local address 127.31.63.126:42219)
0809 08:43:09.742220 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:09.742222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:09.742291 (+ 69us) server_negotiation.cc:407] Connection header received
0809 08:43:09.742362 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:09.742363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:09.742374 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:09.742417 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:09.742754 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.743273 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.743964 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.744152 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.744218 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:09.746877 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:49769: BlockingRecv error: recv got EOF from 127.31.63.66:49769 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 08:43:09 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:09 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:09.879778 13031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:09.875000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:48183)
0809 08:43:09.875174 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:09.875186 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:09.875253 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:09.875397 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:09.875397 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:09.875611 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:09.875665 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:09.875667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.876321 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.876322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:09.876868 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:09.876870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.877283 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.877284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:09.877325 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:09.877845 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:09.877850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:09.879706 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
W20260809 08:43:09.879984 13028 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:09.875098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:48183 (local address 127.31.63.126:42219)
0809 08:43:09.875148 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:09.875150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:09.875265 (+ 115us) server_negotiation.cc:407] Connection header received
0809 08:43:09.875313 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:09.875313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:09.875326 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:09.875380 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:09.875739 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.876248 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.876973 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:09.877197 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:09.877271 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:09.879892 (+ 2621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:48183: BlockingRecv error: recv got EOF from 127.31.63.67:48183 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 09 08:43:10 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:10 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:10.617813 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:10.612599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:57489)
0809 08:43:10.612775 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:10.612786 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:10.612848 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:10.612999 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:10.613000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:10.613172 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:10.613227 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:10.613228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.614010 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.614011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:10.614557 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:10.614559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.614937 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.614938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:10.614988 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:10.615523 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:10.615528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:10.617747 (+ 2219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
W20260809 08:43:10.617935 13034 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:10.612690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:57489 (local address 127.31.63.126:42219)
0809 08:43:10.612804 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:10.612806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:10.612837 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:43:10.612910 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:10.612911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:10.612923 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:10.612984 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:10.613337 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.613902 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.614655 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.614858 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.614926 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:10.617856 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:57489: BlockingRecv error: recv got EOF from 127.31.63.65:57489 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
Aug 09 08:43:10 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:10 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:10.753386 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:10.747325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:36885)
0809 08:43:10.747500 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:10.747511 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:10.747571 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:10.747753 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:10.747755 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:10.747956 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:10.748015 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:10.748017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.748812 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.748814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:10.749927 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:10.749930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.750400 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.750401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:10.750472 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:10.751140 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:10.751148 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:10.753319 (+ 2171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
W20260809 08:43:10.753602 13034 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:10.747373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:36885 (local address 127.31.63.126:42219)
0809 08:43:10.747429 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:10.747432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:10.747573 (+ 141us) server_negotiation.cc:407] Connection header received
0809 08:43:10.747629 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:10.747630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:10.747642 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:10.747684 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:10.748138 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.748724 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.750052 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.750311 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.750378 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:10.753487 (+ 3109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:36885: BlockingRecv error: recv got EOF from 127.31.63.66:36885 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 08:43:10 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:10 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:10.885461 13036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:10.880320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:49909)
0809 08:43:10.880492 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:10.880507 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:10.880581 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:10.880776 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:10.880777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:10.880970 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:10.881023 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:10.881024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.881814 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.881815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:10.882363 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:10.882364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.882745 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.882746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:10.882815 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:10.883361 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:10.883366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:10.885397 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
W20260809 08:43:10.885680 13034 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:10.880457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:49909 (local address 127.31.63.126:42219)
0809 08:43:10.880533 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:10.880535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:10.880560 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:43:10.880669 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:10.880670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:10.880686 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:10.880752 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:10.881147 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.881710 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.882456 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:10.882665 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:10.882746 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:10.885602 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:49909: BlockingRecv error: recv got EOF from 127.31.63.67:49909 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 08:43:11 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:11 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:11.623476 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:11.618445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:40933)
0809 08:43:11.618646 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:11.618662 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:11.618730 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:11.618908 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:11.618909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:11.619083 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:11.619148 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:11.619150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.619813 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.619815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:11.620390 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:11.620392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.620739 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.620740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:11.620787 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:11.621478 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:11.621483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:11.623416 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":97,"threads_started":1}
W20260809 08:43:11.623621 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:11.618534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:40933 (local address 127.31.63.126:42219)
0809 08:43:11.618715 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:11.618717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:11.618726 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:43:11.618808 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:11.618809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:11.618820 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:11.618882 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:11.619212 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.619715 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.620476 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.620665 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.620717 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:11.623523 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:40933: BlockingRecv error: recv got EOF from 127.31.63.65:40933 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":107,"threads_started":1}
Aug 09 08:43:11 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:11 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:11.758452 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:11.753812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:59901)
0809 08:43:11.753992 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:11.754004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:11.754069 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:11.754182 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:11.754183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:11.754354 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:11.754407 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:11.754409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.755049 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.755050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:11.755640 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:11.755642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.755967 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.755968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:11.756011 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:11.756562 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:11.756567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:11.758392 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":110,"threads_started":1}
W20260809 08:43:11.758620 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:11.753890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:59901 (local address 127.31.63.126:42219)
0809 08:43:11.753962 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:11.753964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:11.754065 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:43:11.754115 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:11.754115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:11.754127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:11.754176 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:11.754463 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.754971 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.755711 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.755904 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.755954 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:11.758529 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:59901: BlockingRecv error: recv got EOF from 127.31.63.66:59901 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 08:43:11 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:11 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:11.890661 13042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:11.885971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:43123)
0809 08:43:11.886138 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:11.886149 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:11.886211 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:11.886355 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:11.886356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:11.886518 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:11.886573 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:11.886574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.887226 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.887227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:11.887764 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:11.887766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.888146 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.888147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:11.888191 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:11.888676 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:11.888680 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:11.890585 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
W20260809 08:43:11.890735 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:11.886054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:43123 (local address 127.31.63.126:42219)
0809 08:43:11.886146 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:11.886149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:11.886213 (+ 64us) server_negotiation.cc:407] Connection header received
0809 08:43:11.886278 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:11.886279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:11.886292 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:11.886335 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:11.886646 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.887156 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.887863 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:11.888067 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:11.888138 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:11.890685 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:43123: BlockingRecv error: recv got EOF from 127.31.63.67:43123 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 08:43:12 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:12 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:12.633870 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:12.624105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:57007)
0809 08:43:12.624279 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:12.624297 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:12.624350 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:12.624495 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:12.624496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:12.624656 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:12.624712 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:12.624713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.625361 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.625362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:12.625932 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:12.625934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.626291 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.626292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:12.626339 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:12.631492 (+ 5153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:12.631498 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:12.633755 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
W20260809 08:43:12.633948 13045 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:12.624190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:57007 (local address 127.31.63.126:42219)
0809 08:43:12.624332 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:12.624334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:12.624354 (+ 20us) server_negotiation.cc:407] Connection header received
0809 08:43:12.624409 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:12.624409 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:12.624420 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:12.624484 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:12.624771 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.625285 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.626001 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.626216 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.626270 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:12.633863 (+ 7593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:57007: BlockingRecv error: recv got EOF from 127.31.63.65:57007 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
Aug 09 08:43:12 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:12 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:12.763767 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:12.759135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:54259)
0809 08:43:12.759304 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:12.759317 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:12.759387 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:12.759502 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:12.759503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:12.759688 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:12.759741 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:12.759743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.760436 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.760437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:12.760983 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:12.760985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.761331 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.761332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:12.761374 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:12.761832 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:12.761837 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:12.763699 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
W20260809 08:43:12.763918 13045 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:12.759190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:54259 (local address 127.31.63.126:42219)
0809 08:43:12.759251 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:12.759254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:12.759385 (+ 131us) server_negotiation.cc:407] Connection header received
0809 08:43:12.759433 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:12.759434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:12.759446 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:12.759498 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:12.759802 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.760362 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.761059 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.761265 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.761318 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:12.763816 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:54259: BlockingRecv error: recv got EOF from 127.31.63.66:54259 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 08:43:12 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:12 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:12.895985 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:12.891305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:35175)
0809 08:43:12.891454 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:12.891466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:12.891525 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:12.891661 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:12.891661 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:12.891849 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:12.891905 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:12.891907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.892614 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.892615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:12.893152 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:12.893153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.893515 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.893516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:12.893621 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:12.894103 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:12.894107 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:12.895923 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
W20260809 08:43:12.896227 13045 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:12.891349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:35175 (local address 127.31.63.126:42219)
0809 08:43:12.891451 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:12.891454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:12.891526 (+ 72us) server_negotiation.cc:407] Connection header received
0809 08:43:12.891569 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:12.891570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:12.891585 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:12.891655 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:12.891971 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.892540 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.893236 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:12.893444 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:12.893504 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:12.896153 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:35175: BlockingRecv error: recv got EOF from 127.31.63.67:35175 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 08:43:13 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:13 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:13.639117 13050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:13.634443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:46719)
0809 08:43:13.634617 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:13.634628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:13.634731 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:13.634920 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:13.634921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:13.635098 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:13.635165 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:13.635166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.635828 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.635828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:13.636386 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:13.636388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.636731 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.636732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:13.636776 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:13.637247 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:13.637252 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:13.639054 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
W20260809 08:43:13.639250 13052 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:13.634521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:46719 (local address 127.31.63.126:42219)
0809 08:43:13.634662 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:13.634665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:13.634711 (+ 46us) server_negotiation.cc:407] Connection header received
0809 08:43:13.634799 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:13.634800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:13.634814 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:13.634917 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:13.635235 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.635753 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.636465 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.636662 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.636715 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:13.639163 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:46719: BlockingRecv error: recv got EOF from 127.31.63.65:46719 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
Aug 09 08:43:13 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:13 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:13.769160 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:13.764345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:57435)
0809 08:43:13.764499 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:13.764514 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:13.764592 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:13.764775 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:13.764777 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:13.764971 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:13.765024 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:13.765025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.765796 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.765797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:13.766344 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:13.766346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.766732 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.766733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:13.766803 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:13.767252 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:13.767261 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:13.769099 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
W20260809 08:43:13.769323 13052 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:13.764473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:57435 (local address 127.31.63.126:42219)
0809 08:43:13.764541 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:13.764543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:13.764571 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:43:13.764676 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:13.764677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:13.764695 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:13.764766 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:13.765116 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.765711 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.766445 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.766653 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.766732 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:13.769235 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:57435: BlockingRecv error: recv got EOF from 127.31.63.66:57435 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 08:43:13 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:13 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:13.901579 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:13.896509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:41679)
0809 08:43:13.896670 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:13.896689 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:13.896749 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:13.896881 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:13.896882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:13.897070 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:13.897135 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:13.897136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.897892 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.897893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:13.898466 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:13.898468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.898874 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.898875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:13.898920 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:13.899387 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:13.899392 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:13.901491 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
W20260809 08:43:13.901749 13052 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:13.896628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:41679 (local address 127.31.63.126:42219)
0809 08:43:13.896684 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:13.896686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:13.896736 (+ 50us) server_negotiation.cc:407] Connection header received
0809 08:43:13.896795 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:13.896796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:13.896808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:13.896863 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:13.897211 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.897792 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.898542 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:13.898807 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:13.898861 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:13.901659 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:41679: BlockingRecv error: recv got EOF from 127.31.63.67:41679 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 08:43:14 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:14 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:14.648200 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:14.639675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:34309)
0809 08:43:14.639820 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:14.639840 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:14.639893 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:14.640163 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:14.640164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:14.640359 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:14.640415 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:14.640417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.641116 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.641117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:14.641716 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:14.641719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.642122 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.642123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:14.642165 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:14.646194 (+ 4029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:14.646199 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:14.648139 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
W20260809 08:43:14.648422 13060 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:14.639815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:34309 (local address 127.31.63.126:42219)
0809 08:43:14.639979 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:14.639981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:14.639991 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:43:14.640017 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:14.640017 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:14.640032 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:14.640147 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:14.640493 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.641012 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.641817 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.642020 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.642116 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:14.648306 (+ 6190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:34309: BlockingRecv error: recv got EOF from 127.31.63.65:34309 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
Aug 09 08:43:14 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:14 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:14.774902 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:14.769691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:48047)
0809 08:43:14.769919 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:14.769933 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:14.770038 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:14.770253 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:14.770253 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:14.770453 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:14.770522 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:14.770524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.771217 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.771218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:14.771775 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:14.771777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.772192 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.772193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:14.772257 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:14.772796 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:14.772801 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:14.774803 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":114,"threads_started":1}
W20260809 08:43:14.775007 13060 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:14.769799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:48047 (local address 127.31.63.126:42219)
0809 08:43:14.769902 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:14.769905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:14.770017 (+ 112us) server_negotiation.cc:407] Connection header received
0809 08:43:14.770148 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:14.770149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:14.770163 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:14.770236 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:14.770604 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.771127 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.771877 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.772077 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.772192 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:14.774929 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:48047: BlockingRecv error: recv got EOF from 127.31.63.66:48047 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 08:43:14 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:14 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:14.907612 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:14.902127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:60109)
0809 08:43:14.902306 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:14.902319 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:14.902419 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:14.902625 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:14.902626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:14.902836 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:14.902891 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:14.902893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.903660 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.903662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:14.904285 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:14.904287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.904726 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.904727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:14.904804 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:14.905482 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:14.905489 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:14.907543 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":62,"threads_started":1}
W20260809 08:43:14.907717 13060 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:14.902329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:60109 (local address 127.31.63.126:42219)
0809 08:43:14.902407 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:14.902409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:14.902421 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:43:14.902505 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:14.902506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:14.902527 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:14.902600 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:14.902987 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.903548 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.904406 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:14.904635 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:14.904725 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:14.907644 (+ 2919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:60109: BlockingRecv error: recv got EOF from 127.31.63.67:60109 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 08:43:15 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:15 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:15.654143 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:15.648801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:47157)
0809 08:43:15.648991 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:15.649004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:15.649088 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:15.649285 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:15.649286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:15.649473 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:15.649530 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:15.649531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.650268 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.650269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:15.650907 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:15.650909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.651279 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.651280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:15.651398 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:15.652029 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:15.652034 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:15.654040 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
W20260809 08:43:15.654214 13066 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:15.648877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:47157 (local address 127.31.63.126:42219)
0809 08:43:15.649004 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:15.649008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:15.649073 (+ 65us) server_negotiation.cc:407] Connection header received
0809 08:43:15.649149 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:15.649150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:15.649161 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:15.649225 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:15.649628 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.650154 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.650982 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.651177 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.651231 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:15.654158 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:47157: BlockingRecv error: recv got EOF from 127.31.63.65:47157 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
Aug 09 08:43:15 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:15 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:15.780145 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:15.775389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:48887)
0809 08:43:15.775544 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:15.775554 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:15.775617 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:15.775768 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:15.775769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:15.775945 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:15.775999 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:15.776000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.776638 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.776639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:15.777418 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:15.777420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.777752 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.777753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:15.777796 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:15.778223 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:15.778227 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:15.780058 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
W20260809 08:43:15.780203 13066 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:15.775450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:48887 (local address 127.31.63.126:42219)
0809 08:43:15.775531 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:15.775533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:15.775622 (+ 89us) server_negotiation.cc:407] Connection header received
0809 08:43:15.775678 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:15.775679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:15.775691 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:15.775747 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:15.776066 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.776570 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.777474 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.777687 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.777743 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:15.780156 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:48887: BlockingRecv error: recv got EOF from 127.31.63.66:48887 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 08:43:15 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:15 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:15.913122 13069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:15.908227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:57263)
0809 08:43:15.908451 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:15.908464 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:15.908538 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:15.908696 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:15.908697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:15.908897 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:15.908949 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:15.908950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.909702 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.909703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:15.910263 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:15.910265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.910643 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.910644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:15.910712 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:15.911165 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:15.911170 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:15.913023 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":116,"threads_started":1}
W20260809 08:43:15.913177 13066 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:15.908323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:57263 (local address 127.31.63.126:42219)
0809 08:43:15.908432 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:15.908434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:15.908531 (+ 97us) server_negotiation.cc:407] Connection header received
0809 08:43:15.908595 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:15.908595 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:15.908610 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:15.908688 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:15.909033 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.909620 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.910358 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:15.910555 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:15.910628 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:15.913107 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:57263: BlockingRecv error: recv got EOF from 127.31.63.67:57263 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 08:43:16 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:16 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:16.659703 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:16.654758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:36825)
0809 08:43:16.654928 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:16.654944 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:16.655020 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:16.655238 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:16.655239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:16.655414 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:16.655472 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:16.655474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.656148 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.656150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:16.656724 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:16.656726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.657080 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.657081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:16.657125 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:16.657707 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:16.657712 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:16.659640 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
W20260809 08:43:16.659911 13071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:16.654899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:36825 (local address 127.31.63.126:42219)
0809 08:43:16.655081 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:16.655084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:16.655092 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:43:16.655118 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:16.655118 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:16.655129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:16.655215 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:16.655545 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.656053 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.656809 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.657009 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.657063 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:16.659837 (+ 2774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:36825: BlockingRecv error: recv got EOF from 127.31.63.65:36825 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":106,"threads_started":1}
Aug 09 08:43:16 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:16 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:16.785616 13072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:16.780677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:46057)
0809 08:43:16.780876 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:16.780888 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:16.780966 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:16.781143 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:16.781144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:16.781410 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:16.781480 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:16.781483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.782343 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.782344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:16.782866 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:16.782867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.783250 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.783250 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:16.783305 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:16.783760 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:16.783766 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:16.785491 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":111,"threads_started":1}
W20260809 08:43:16.785725 13071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:16.780753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:46057 (local address 127.31.63.126:42219)
0809 08:43:16.780851 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:16.780854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:16.780964 (+ 110us) server_negotiation.cc:407] Connection header received
0809 08:43:16.781026 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:16.781030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:16.781046 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:16.781108 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:16.781583 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.782266 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.782963 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.783176 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.783240 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:16.785651 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:46057: BlockingRecv error: recv got EOF from 127.31.63.66:46057 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 08:43:16 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:16 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:16.918363 13073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:16.913617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:55185)
0809 08:43:16.913764 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:16.913777 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:16.913851 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:16.913996 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:16.913996 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:16.914166 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:16.914224 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:16.914226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.914873 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.914874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:16.915412 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:16.915414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.915762 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.915763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:16.915820 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:16.916408 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:16.916418 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:16.918300 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
W20260809 08:43:16.918540 13071 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:16.913709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:55185 (local address 127.31.63.126:42219)
0809 08:43:16.913801 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:16.913808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:16.913831 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:43:16.913910 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:16.913911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:16.913924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:16.913974 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:16.914288 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.914791 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.915488 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:16.915676 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:16.915728 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:16.918469 (+ 2741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:55185: BlockingRecv error: recv got EOF from 127.31.63.67:55185 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 08:43:17 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:17 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:17.671239 13074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:17.660321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:38817)
0809 08:43:17.660465 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:17.660479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:17.660550 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:17.660725 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:17.660725 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:17.660925 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:17.660981 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:17.660983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.661743 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.661744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:17.662297 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:17.662298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.662711 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.662712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:17.662756 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:17.668921 (+ 6165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:17.668927 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:17.671167 (+ 2240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
W20260809 08:43:17.671411 13075 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:17.660439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:38817 (local address 127.31.63.126:42219)
0809 08:43:17.660580 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:17.660582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:17.660591 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:43:17.660621 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:17.660622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:17.660633 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:17.660705 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:17.661068 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.661667 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.662399 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.662633 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.662705 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:17.671307 (+ 8602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:38817: BlockingRecv error: recv got EOF from 127.31.63.65:38817 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
Aug 09 08:43:17 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:17 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:17.790987 13076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:17.786080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:56665)
0809 08:43:17.786272 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:17.786289 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:17.786372 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:17.786534 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:17.786535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:17.786731 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:17.786784 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:17.786786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.787475 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.787477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:17.788067 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:17.788068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.788454 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.788455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:17.788522 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:17.789025 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:17.789030 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:17.790890 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":87,"threads_started":1}
W20260809 08:43:17.791038 13075 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:17.786159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:56665 (local address 127.31.63.126:42219)
0809 08:43:17.786251 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:17.786253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:17.786351 (+ 98us) server_negotiation.cc:407] Connection header received
0809 08:43:17.786453 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:17.786454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:17.786468 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:17.786515 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:17.786863 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.787377 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.788154 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.788369 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.788448 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:17.790970 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:56665: BlockingRecv error: recv got EOF from 127.31.63.66:56665 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 08:43:17 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:17 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:17.923554 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:17.918862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:54843)
0809 08:43:17.919040 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:17.919052 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:17.919114 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:17.919264 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:17.919265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:17.919441 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:17.919494 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:17.919496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.920148 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.920148 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:17.920713 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:17.920715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.921051 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.921052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:17.921094 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:17.921601 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:17.921605 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:17.923483 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
W20260809 08:43:17.923709 13075 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:17.918942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:54843 (local address 127.31.63.126:42219)
0809 08:43:17.919031 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:17.919033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:17.919112 (+ 79us) server_negotiation.cc:407] Connection header received
0809 08:43:17.919157 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:17.919158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:17.919171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:17.919241 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:17.919559 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.920076 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.920786 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:17.920981 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:17.921037 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:17.923612 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:54843: BlockingRecv error: recv got EOF from 127.31.63.67:54843 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 08:43:18 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:18 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:18.676411 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:18.671809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:54007)
0809 08:43:18.671968 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:18.671979 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:18.672030 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:18.672166 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:18.672166 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:18.672341 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:18.672397 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:18.672398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.673059 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.673060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:18.673670 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:18.673672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.674008 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.674009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:18.674053 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:18.674577 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:18.674582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:18.676351 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
W20260809 08:43:18.676579 13080 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:18.671871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:54007 (local address 127.31.63.126:42219)
0809 08:43:18.672027 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:18.672029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:18.672039 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:43:18.672071 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:18.672071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:18.672083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:18.672145 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:18.672466 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.672986 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.673743 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.673947 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.673996 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:18.676481 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:54007: BlockingRecv error: recv got EOF from 127.31.63.65:54007 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
Aug 09 08:43:18 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:18 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:18.796378 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:18.791473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:56031)
0809 08:43:18.791615 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:18.791633 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:18.791692 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:18.791836 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:18.791836 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:18.792010 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:18.792066 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:18.792067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.792696 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.792698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:18.793263 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:18.793266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.793619 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.793620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:18.793668 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:18.794180 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:18.794185 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:18.796249 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
W20260809 08:43:18.796406 13080 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:18.791549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:56031 (local address 127.31.63.126:42219)
0809 08:43:18.791619 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:18.791622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:18.791693 (+ 71us) server_negotiation.cc:407] Connection header received
0809 08:43:18.791752 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:18.791753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:18.791776 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:18.791818 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:18.792128 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.792621 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.793337 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.793521 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.793588 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:18.796340 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:56031: BlockingRecv error: recv got EOF from 127.31.63.66:56031 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 08:43:18 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:18 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:18.928944 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:18.924107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:56283)
0809 08:43:18.924301 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:18.924316 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:18.924392 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:18.924525 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:18.924526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:18.924721 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:18.924782 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:18.924783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.925500 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.925501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:18.926079 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:18.926081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.926480 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.926481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:18.926543 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:18.927005 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:18.927010 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:18.928883 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":119,"threads_started":1}
W20260809 08:43:18.929102 13080 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:18.924243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:56283 (local address 127.31.63.126:42219)
0809 08:43:18.924326 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:18.924328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:18.924371 (+ 43us) server_negotiation.cc:407] Connection header received
0809 08:43:18.924449 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:18.924450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:18.924465 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:18.924519 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:18.924871 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.925400 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.926176 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:18.926398 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:18.926481 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:18.929002 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:56283: BlockingRecv error: recv got EOF from 127.31.63.67:56283 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 08:43:19 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:19 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:19.681965 13083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:19.676935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:49681)
0809 08:43:19.677096 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:19.677107 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:19.677158 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:19.677306 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:19.677306 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:19.677483 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:19.677561 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:19.677563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.678325 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.678326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:19.678870 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:19.678872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.679335 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.679336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:19.679384 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:19.679840 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:19.679844 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:19.681851 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
W20260809 08:43:19.682010 13084 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:19.676992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:49681 (local address 127.31.63.126:42219)
0809 08:43:19.677147 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:19.677149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:19.677159 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:43:19.677218 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:19.677219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:19.677229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:19.677295 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:19.677637 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.678253 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.678996 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.679261 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.679333 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:19.681935 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:49681: BlockingRecv error: recv got EOF from 127.31.63.65:49681 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
Aug 09 08:43:19 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:19 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:19.802650 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:19.796943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:39063)
0809 08:43:19.797128 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:19.797146 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:19.797245 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:19.797415 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:19.797416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:19.797672 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:19.797740 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:19.797742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.798617 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.798618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:19.799281 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:19.799284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.799661 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.799662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:19.799710 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:19.800303 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:19.800310 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:19.802572 (+ 2262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
W20260809 08:43:19.802853 13084 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:19.797090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:39063 (local address 127.31.63.126:42219)
0809 08:43:19.797199 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:19.797201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:19.797246 (+ 45us) server_negotiation.cc:407] Connection header received
0809 08:43:19.797296 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:19.797297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:19.797309 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:19.797362 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:19.797891 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.798533 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.799376 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.799588 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.799655 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:19.802777 (+ 3122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:39063: BlockingRecv error: recv got EOF from 127.31.63.66:39063 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 08:43:19 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:19 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:19.934578 13086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:19.929517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:37203)
0809 08:43:19.929746 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:19.929757 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:19.929828 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:19.929983 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:19.929984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:19.930169 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:19.930223 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:19.930224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.930961 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.930963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:19.931738 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:19.931742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.932082 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.932083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:19.932129 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:19.932578 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:19.932583 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:19.934478 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":95,"threads_started":1}
W20260809 08:43:19.934628 13084 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:19.929666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:37203 (local address 127.31.63.126:42219)
0809 08:43:19.929742 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:19.929745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:19.929827 (+ 82us) server_negotiation.cc:407] Connection header received
0809 08:43:19.929888 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:19.929889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:19.929903 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:19.929945 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:19.930310 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.930858 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.931817 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:19.932011 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:19.932065 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:19.934561 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:37203: BlockingRecv error: recv got EOF from 127.31.63.67:37203 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 08:43:20 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:20 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:20.688208 13087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:20.682556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:60325)
0809 08:43:20.682762 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:20.682780 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:20.682874 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:20.683065 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:20.683066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:20.683306 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:20.683362 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:20.683364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.684207 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.684209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:20.684839 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:20.684841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.685287 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.685288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:20.685364 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:20.685965 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:20.685972 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:20.688074 (+ 2102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":98,"threads_started":1}
W20260809 08:43:20.688269 13089 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:20.682631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:60325 (local address 127.31.63.126:42219)
0809 08:43:20.682815 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:20.682817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:20.682850 (+ 33us) server_negotiation.cc:407] Connection header received
0809 08:43:20.682936 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:20.682937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:20.682948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:20.683012 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:20.683497 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.684107 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.684950 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.685187 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.685284 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:20.688182 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:60325: BlockingRecv error: recv got EOF from 127.31.63.65:60325 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":120,"threads_started":1}
Aug 09 08:43:20 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:20 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:20.808750 13090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:20.803288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:48431)
0809 08:43:20.803540 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:20.803555 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:20.803671 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:20.803827 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:20.803828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:20.804048 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:20.804100 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:20.804102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.804881 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.804883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:20.805500 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:20.805502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.806013 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.806015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:20.806096 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:20.806730 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:20.806735 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:20.808648 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":106,"threads_started":1}
W20260809 08:43:20.808799 13089 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:20.803415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:48431 (local address 127.31.63.126:42219)
0809 08:43:20.803523 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:20.803526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:20.803651 (+ 125us) server_negotiation.cc:407] Connection header received
0809 08:43:20.803750 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:20.803751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:20.803765 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:20.803819 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:20.804192 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.804779 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.805673 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.805911 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.806010 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:20.808732 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:48431: BlockingRecv error: recv got EOF from 127.31.63.66:48431 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 08:43:20 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:20 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:20.939803 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:20.935050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:57207)
0809 08:43:20.935238 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:20.935251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:20.935323 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:20.935473 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:20.935473 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:20.935671 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:20.935723 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:20.935725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.936411 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.936413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:20.936967 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:20.936969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.937339 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.937340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:20.937394 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:20.937958 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:20.937964 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:20.939745 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":140,"threads_started":1}
W20260809 08:43:20.939975 13089 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:20.935180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:57207 (local address 127.31.63.126:42219)
0809 08:43:20.935233 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:20.935236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:20.935362 (+ 126us) server_negotiation.cc:407] Connection header received
0809 08:43:20.935390 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:20.935390 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:20.935405 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:20.935453 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:20.935794 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.936327 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.937043 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:20.937239 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:20.937295 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:20.939872 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:57207: BlockingRecv error: recv got EOF from 127.31.63.67:57207 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 08:43:21 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:21 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:21.694094 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:21.688844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:57791)
0809 08:43:21.689015 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:21.689026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:21.689084 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:21.689274 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:21.689275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:21.689455 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:21.689509 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:21.689511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.690222 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.690222 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:21.690751 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:21.690753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.691128 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.691129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:21.691171 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:21.691640 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:21.691644 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:21.694033 (+ 2389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
W20260809 08:43:21.694284 13094 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:21.688956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:57791 (local address 127.31.63.126:42219)
0809 08:43:21.689132 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:21.689135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:21.689144 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:43:21.689171 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:21.689171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:21.689181 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:21.689244 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:21.689622 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.690153 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.690831 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.691030 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.691124 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:21.694189 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:57791: BlockingRecv error: recv got EOF from 127.31.63.65:57791 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
Aug 09 08:43:21 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:21 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:21.815039 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:21.809592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:45661)
0809 08:43:21.809807 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:21.809824 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:21.809898 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:21.810042 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:21.810043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:21.810265 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:21.810321 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:21.810323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.810972 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.810973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:21.811625 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:21.811627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.812035 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.812037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:21.812149 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:21.812811 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:21.812817 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:21.814926 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":101,"threads_started":1}
W20260809 08:43:21.815091 13094 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:21.809742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:45661 (local address 127.31.63.126:42219)
0809 08:43:21.809821 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:21.809823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:21.809880 (+ 57us) server_negotiation.cc:407] Connection header received
0809 08:43:21.809949 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:21.809950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:21.809962 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:21.810012 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:21.810391 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.810892 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.811704 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.811955 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.812009 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:21.815019 (+ 3010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:45661: BlockingRecv error: recv got EOF from 127.31.63.66:45661 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 08:43:21 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:21 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:21.945575 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:21.940426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:54767)
0809 08:43:21.940636 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:21.940648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:21.940720 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:21.940909 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:21.940910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:21.941107 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:21.941161 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:21.941163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.941867 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.941868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:21.942416 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:21.942419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.942739 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.942740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:21.942783 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:21.943328 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:21.943333 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:21.945446 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":111,"threads_started":1}
W20260809 08:43:21.945618 13094 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:21.940496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:54767 (local address 127.31.63.126:42219)
0809 08:43:21.940571 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:21.940574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:21.940715 (+ 141us) server_negotiation.cc:407] Connection header received
0809 08:43:21.940777 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:21.940778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:21.940793 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:21.940856 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:21.941233 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.941783 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.942492 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:21.942673 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:21.942723 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:21.945552 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:54767: BlockingRecv error: recv got EOF from 127.31.63.67:54767 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 08:43:22 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:22 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:22.703152 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:22.694648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:46889)
0809 08:43:22.694858 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:22.694870 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:22.694935 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:22.695162 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:22.695163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:22.695364 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:22.695425 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:22.695427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.696146 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.696147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:22.696743 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:22.696744 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.697239 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.697241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:22.697305 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:22.701008 (+ 3703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:22.701014 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:22.703054 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":126,"threads_started":1}
W20260809 08:43:22.703220 13099 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:22.694726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:46889 (local address 127.31.63.126:42219)
0809 08:43:22.694881 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:22.694883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:22.694918 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:43:22.694983 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:22.694984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:22.694995 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:22.695065 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:22.695514 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.696025 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.696856 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.697149 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.697204 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:22.703164 (+ 5960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:46889: BlockingRecv error: recv got EOF from 127.31.63.65:46889 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
Aug 09 08:43:22 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:22 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:22.820895 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:22.815721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:42723)
0809 08:43:22.815899 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:22.815918 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:22.815976 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:22.816111 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:22.816112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:22.816289 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:22.816342 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:22.816343 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.817054 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.817055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:22.817932 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:22.817935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.818306 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.818307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:22.818348 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:22.818844 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:22.818849 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:22.820796 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":119,"threads_started":1}
W20260809 08:43:22.820948 13099 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:22.815823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:42723 (local address 127.31.63.126:42219)
0809 08:43:22.815878 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:22.815880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:22.815961 (+ 81us) server_negotiation.cc:407] Connection header received
0809 08:43:22.816024 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:22.816025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:22.816036 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:22.816089 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:22.816411 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.816966 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.818022 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.818236 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.818298 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:22.820884 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:42723: BlockingRecv error: recv got EOF from 127.31.63.66:42723 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 08:43:22 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:22 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.67@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:22.951814 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:22.946184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.67:53023)
0809 08:43:22.946380 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:22.946392 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:22.946491 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:22.946695 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:22.946696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:22.946901 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:22.946953 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:22.946955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.947682 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.947684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:22.948255 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:22.948257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.948639 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.948640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:22.948707 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:22.949419 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:22.949426 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:22.951740 (+ 2314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":98,"threads_started":1}
W20260809 08:43:22.952032 13099 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:22.946277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:53023 (local address 127.31.63.126:42219)
0809 08:43:22.946371 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:22.946374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:22.946469 (+ 95us) server_negotiation.cc:407] Connection header received
0809 08:43:22.946595 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:22.946597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:22.946617 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:22.946676 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:22.947050 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.947593 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.948343 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:22.948558 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:22.948640 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:22.951964 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.67:53023: BlockingRecv error: recv got EOF from 127.31.63.67:53023 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 08:43:23 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:23 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.65@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:23.708855 13102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:23.703729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.65:33305)
0809 08:43:23.703989 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:23.704002 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:23.704071 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:23.704245 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:23.704247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:23.704440 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:23.704494 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:23.704496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:23.705202 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:23.705203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:23.705794 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:23.705796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:23.706210 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:23.706211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:23.706281 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:23.706791 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:23.706796 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:23.708742 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":109,"threads_started":1}
W20260809 08:43:23.708914 13105 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:23.703834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:33305 (local address 127.31.63.126:42219)
0809 08:43:23.703975 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:23.703977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:23.704078 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:43:23.704115 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:23.704116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:23.704137 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:23.704225 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:23.704584 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:23.705111 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:23.705888 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:23.706100 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:23.706208 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:23.708835 (+ 2627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.65:33305: BlockingRecv error: recv got EOF from 127.31.63.65:33305 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
Aug 09 08:43:23 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:43:23 dist-test-slave-gkw7 krb5kdc[11548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.63.66@KRBTEST.COM for oryx/127.31.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:43:23.826814 13106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:23.821579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:42219 (local address 127.31.63.66:53809)
0809 08:43:23.821734 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:23.821747 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:23.821847 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:23.821998 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:23.821998 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:23.822173 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:23.822227 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:23.822228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:23.823043 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:23.823045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:23.823741 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:23.823745 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:23.824119 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:23.824120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:23.824186 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:23.824662 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:23.824667 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:23.826736 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.63.126:42219: Server oryx/127.31.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
W20260809 08:43:23.826937 13105 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:23.821634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:53809 (local address 127.31.63.126:42219)
0809 08:43:23.821736 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:23.821739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:23.821848 (+ 109us) server_negotiation.cc:407] Connection header received
0809 08:43:23.821897 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:23.821898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:23.821909 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:23.821973 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:23.822294 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:23.822950 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:23.823828 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:23.824021 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:23.824078 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:23.826838 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.63.66:53809: BlockingRecv error: recv got EOF from 127.31.63.66:53809 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 08:43:23.865870 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 12421
I20260809 08:43:23.879554 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 12640
I20260809 08:43:23.887989 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 12835
I20260809 08:43:23.899341 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 12316
2026-08-09T08:43:23Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17852 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T08:43:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:43:23Z Disabled control of system clock
I20260809 08:43:23.935742 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:46303
--webserver_interface=127.31.63.126
--webserver_port=0
--builtin_ntp_servers=127.31.63.84:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:46303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 08:43:24.018872 13112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:24.019088 13112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:24.019109 13112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:24.020407 13112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:43:24.020455 13112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:24.020469 13112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:24.020483 13112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:43:24.020494 13112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:43:24.021981 13112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:46303
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:46303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:24.022234 13112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:24.022567 13112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:24.027987 13126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.027987 13122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:24.028043 13112 server_base.cc:1034] running on GCE node
W20260809 08:43:24.027992 13121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:24.028503 13112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:24.028862 13112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:24.030005 13112 hybrid_clock.cc:648] HybridClock initialized: now 1786265004029979 us; error 51 us; skew 500 ppm
I20260809 08:43:24.031234 13112 webserver.cc:460] Webserver started at http://127.31.63.126:35167/ using document root <none> and password file <none>
I20260809 08:43:24.031491 13112 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:24.031539 13112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:24.031673 13112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:24.032755 13112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "c56ce40c4a6547fe8045c36e9acf2c5b"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "6d60848f2efbdeb475a6c9e4d413e037"
server_key_iv: "e4e65611c84821c8ce0b0e4dec66ff47"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.033130 13112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "c56ce40c4a6547fe8045c36e9acf2c5b"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "6d60848f2efbdeb475a6c9e4d413e037"
server_key_iv: "e4e65611c84821c8ce0b0e4dec66ff47"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.034564 13112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:43:24.035419 13136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.035661 13112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:43:24.035741 13112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "c56ce40c4a6547fe8045c36e9acf2c5b"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "6d60848f2efbdeb475a6c9e4d413e037"
server_key_iv: "e4e65611c84821c8ce0b0e4dec66ff47"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.035792 13112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:24.045092 13112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:24.045398 13112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:24.045575 13112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:24.049836 13112 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:46303
I20260809 08:43:24.049891 13207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:46303 every 8 connection(s)
I20260809 08:43:24.050268 13112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:43:24.051057 13209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:24.053982 13209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b: Bootstrap starting.
I20260809 08:43:24.054596 13209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:24.054868 13209 log.cc:824] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:24.055584 13209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b: No bootstrap required, opened a new log
I20260809 08:43:24.057219 13209 raft_consensus.cc:348] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ce40c4a6547fe8045c36e9acf2c5b" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46303 } }
I20260809 08:43:24.057371 13209 raft_consensus.cc:374] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:24.057394 13209 raft_consensus.cc:729] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c56ce40c4a6547fe8045c36e9acf2c5b, State: Initialized, Role: FOLLOWER
I20260809 08:43:24.057500 13209 consensus_queue.cc:260] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ce40c4a6547fe8045c36e9acf2c5b" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46303 } }
I20260809 08:43:24.057646 13209 raft_consensus.cc:388] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:43:24.057674 13209 raft_consensus.cc:482] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:43:24.057708 13209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:24.058230 13209 raft_consensus.cc:504] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ce40c4a6547fe8045c36e9acf2c5b" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46303 } }
I20260809 08:43:24.058320 13209 leader_election.cc:302] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c56ce40c4a6547fe8045c36e9acf2c5b; no voters:
I20260809 08:43:24.058449 13209 leader_election.cc:288] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:43:24.058530 13213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:24.058737 13213 raft_consensus.cc:686] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [term 1 LEADER]: Becoming Leader. State: Replica: c56ce40c4a6547fe8045c36e9acf2c5b, State: Running, Role: LEADER
I20260809 08:43:24.058745 13209 sys_catalog.cc:564] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:43:24.058894 13213 consensus_queue.cc:237] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ce40c4a6547fe8045c36e9acf2c5b" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46303 } }
I20260809 08:43:24.059341 13214 sys_catalog.cc:455] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c56ce40c4a6547fe8045c36e9acf2c5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ce40c4a6547fe8045c36e9acf2c5b" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46303 } } }
I20260809 08:43:24.059427 13214 sys_catalog.cc:458] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:24.059458 13217 sys_catalog.cc:455] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c56ce40c4a6547fe8045c36e9acf2c5b. Latest consensus state: current_term: 1 leader_uuid: "c56ce40c4a6547fe8045c36e9acf2c5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ce40c4a6547fe8045c36e9acf2c5b" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46303 } } }
I20260809 08:43:24.059551 13217 sys_catalog.cc:458] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:24.059888 13226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:43:24.060432 13226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:43:24.060434 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 13112
I20260809 08:43:24.060534 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:43:24.060781 31997 external_mini_cluster.cc:1428] Setting key 474aaea504d1f49e5f8ce3cefe39ca1d
I20260809 08:43:24.062201 13226 catalog_manager.cc:1347] Generated new cluster ID: 09804dce9fe14499b978c130e9d23bf7
I20260809 08:43:24.062253 13226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:43:24.066304 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:24.061261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.31.63.126:46303)
0809 08:43:24.061833 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:24.061842 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:24.061867 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:43:24.061962 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:24.061968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:24.062024 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:24.062243 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:24.062447 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.063335 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.064177 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.064615 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.064761 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:24.065524 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:24.065555 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:24.065555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:43:24.065783 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:24.065818 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:43:24.065830 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:24.065913 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:24.065993 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 08:43:24.066076 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":80,"threads_started":1}
I20260809 08:43:24.076301 13226 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:43:24.077001 13226 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:43:24.092831 13226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c56ce40c4a6547fe8045c36e9acf2c5b: Generated new TSK 0
I20260809 08:43:24.093061 13226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:43:24.104013 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46303
--builtin_ntp_servers=127.31.63.84:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 08:43:24.192008 13243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:24.192260 13243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:24.192293 13243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:24.193446 13243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:24.193516 13243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:24.193579 13243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:43:24.194941 13243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:24.195215 13243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:24.195544 13243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:24.200802 13249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:24.200819 13243 server_base.cc:1034] running on GCE node
W20260809 08:43:24.200790 13253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.200790 13250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:24.201205 13243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:24.201471 13243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:24.202639 13243 hybrid_clock.cc:648] HybridClock initialized: now 1786265004202648 us; error 54 us; skew 500 ppm
I20260809 08:43:24.204159 13243 webserver.cc:460] Webserver started at http://127.31.63.65:43033/ using document root <none> and password file <none>
I20260809 08:43:24.204445 13243 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:24.204523 13243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:24.204695 13243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:24.205966 13243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "6b8f8416069a4232a46bb174b4b59ab7"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "296dc02305eade0343ab089034c0fe6c"
server_key_iv: "f864743e909c1aea8bde4c6922484469"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.206369 13243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "6b8f8416069a4232a46bb174b4b59ab7"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "296dc02305eade0343ab089034c0fe6c"
server_key_iv: "f864743e909c1aea8bde4c6922484469"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.207839 13243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 08:43:24.208636 13261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.208832 13243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:43:24.208931 13243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "6b8f8416069a4232a46bb174b4b59ab7"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "296dc02305eade0343ab089034c0fe6c"
server_key_iv: "f864743e909c1aea8bde4c6922484469"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.209017 13243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:24.225718 13243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:24.226042 13243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:24.226199 13243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:24.226447 13243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:24.226806 13243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:24.226864 13243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.226902 13243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:24.226948 13243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.232908 13243 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:46301
I20260809 08:43:24.233013 13420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:46301 every 8 connection(s)
I20260809 08:43:24.233359 13243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:43:24.239097 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:24.234192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:38313 (local address 127.31.63.126:46303)
0809 08:43:24.234246 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:24.234249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:24.234491 (+ 242us) server_negotiation.cc:407] Connection header received
0809 08:43:24.235057 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:24.235058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:24.235076 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:24.235143 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:24.235875 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.236396 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.237448 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.237705 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.237790 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:24.238772 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:24.238774 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:24.238774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:43:24.238788 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:24.238803 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:43:24.238808 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:24.238877 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:24.238981 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 08:43:24.239014 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:43:24.239338 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:24.234155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:46303 (local address 127.31.63.65:38313)
0809 08:43:24.234357 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:24.234392 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:24.234940 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:24.235217 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:24.235221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:24.235232 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:24.235770 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:24.235774 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.236516 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.236517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:24.237346 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:24.237349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.237808 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.237809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:24.237953 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:24.238610 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:43:24.238610 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:24.238640 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:43:24.238664 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:24.238879 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:24.238886 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:24.238890 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:43:24.238982 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:43:24.239092 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260809 08:43:24.239952 13421 heartbeater.cc:347] Connected to a master server at 127.31.63.126:46303
I20260809 08:43:24.240049 13421 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:24.240202 13421 heartbeater.cc:511] Master 127.31.63.126:46303 requested a full tablet report, sending...
I20260809 08:43:24.240582 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 13243
I20260809 08:43:24.240696 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:43:24.240813 13161 ts_manager.cc:194] Registered new tserver with Master: 6b8f8416069a4232a46bb174b4b59ab7 (127.31.63.65:46301)
I20260809 08:43:24.240935 31997 external_mini_cluster.cc:1428] Setting key 0347ea092fc0f429698122ba1eead446
I20260809 08:43:24.241647 13161 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.65:38313
I20260809 08:43:24.242187 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46303
--builtin_ntp_servers=127.31.63.84:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 08:43:24.327564 13426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:24.327776 13426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:24.327800 13426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:24.328976 13426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:24.329017 13426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:24.329042 13426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:43:24.330338 13426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:24.330533 13426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:24.330777 13426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:24.336122 13435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.336159 13434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.336218 13438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:24.336381 13426 server_base.cc:1034] running on GCE node
I20260809 08:43:24.336668 13426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:24.336967 13426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:24.338153 13426 hybrid_clock.cc:648] HybridClock initialized: now 1786265004338121 us; error 49 us; skew 500 ppm
I20260809 08:43:24.339618 13426 webserver.cc:460] Webserver started at http://127.31.63.66:32785/ using document root <none> and password file <none>
I20260809 08:43:24.339871 13426 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:24.339922 13426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:24.340106 13426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:24.341259 13426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "d0e934b935bf4f85a1e2ae89003fb427"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "61539292062f70ebbe0ba01a8db30d82"
server_key_iv: "d94d9a81007676964247d2cd560f7b17"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.341676 13426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "d0e934b935bf4f85a1e2ae89003fb427"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "61539292062f70ebbe0ba01a8db30d82"
server_key_iv: "d94d9a81007676964247d2cd560f7b17"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.343083 13426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:24.343884 13448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.344060 13426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:43:24.344192 13426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "d0e934b935bf4f85a1e2ae89003fb427"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "61539292062f70ebbe0ba01a8db30d82"
server_key_iv: "d94d9a81007676964247d2cd560f7b17"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.344271 13426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:24.352133 13426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:24.352439 13426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:24.352593 13426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:24.352824 13426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:24.353175 13426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:24.353231 13426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.353274 13426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:24.353302 13426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.359517 13426 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:44905
I20260809 08:43:24.359624 13613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:44905 every 8 connection(s)
I20260809 08:43:24.359962 13426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:43:24.365247 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:24.360777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:46039 (local address 127.31.63.126:46303)
0809 08:43:24.360851 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:24.360853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:24.361080 (+ 227us) server_negotiation.cc:407] Connection header received
0809 08:43:24.361663 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:24.361664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:24.361675 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:24.361730 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:24.362405 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.362902 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.363871 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.364057 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.364110 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:24.364986 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:24.364987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:24.364987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:43:24.364991 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:24.365002 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:43:24.365006 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:24.365044 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:24.365135 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 08:43:24.365158 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:43:24.365440 13617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:24.360730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:46303 (local address 127.31.63.66:46039)
0809 08:43:24.360949 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:24.360984 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:24.361485 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:24.361799 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:24.361803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:24.361812 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:24.362324 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:24.362329 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.362988 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.362989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:24.363792 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:24.363794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.364140 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.364141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:24.364256 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:24.364851 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:43:24.364852 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:24.364883 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:43:24.364906 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:24.365061 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:24.365066 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:24.365068 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:43:24.365124 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 08:43:24.365209 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":104,"threads_started":1}
I20260809 08:43:24.365999 13614 heartbeater.cc:347] Connected to a master server at 127.31.63.126:46303
I20260809 08:43:24.366086 13614 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:24.366350 13614 heartbeater.cc:511] Master 127.31.63.126:46303 requested a full tablet report, sending...
I20260809 08:43:24.366781 13161 ts_manager.cc:194] Registered new tserver with Master: d0e934b935bf4f85a1e2ae89003fb427 (127.31.63.66:44905)
I20260809 08:43:24.367125 13161 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.66:46039
I20260809 08:43:24.368027 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 13426
I20260809 08:43:24.368110 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:43:24.368322 31997 external_mini_cluster.cc:1428] Setting key 4b79b8b82c055ac194218a30a79927a8
I20260809 08:43:24.369256 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46303
--builtin_ntp_servers=127.31.63.84:40539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 08:43:24.454196 13618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:24.454408 13618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:24.454465 13618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:24.455603 13618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:24.455644 13618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:24.455669 13618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:43:24.456947 13618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:24.457132 13618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:24.457381 13618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:24.462699 13624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.462741 13626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.462706 13629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:24.463083 13618 server_base.cc:1034] running on GCE node
I20260809 08:43:24.463337 13618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:24.463579 13618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:24.464751 13618 hybrid_clock.cc:648] HybridClock initialized: now 1786265004464737 us; error 32 us; skew 500 ppm
I20260809 08:43:24.466138 13618 webserver.cc:460] Webserver started at http://127.31.63.67:45185/ using document root <none> and password file <none>
I20260809 08:43:24.466398 13618 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:24.466468 13618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:24.466630 13618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:24.467751 13618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "5b080795a66a4f05a810e042f8c28c8c"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "92228c0b7ddde1b1b3cd6fb91939d191"
server_key_iv: "e6dfbcfa3eba288f3b37fdf4d495d6f4"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.468130 13618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "5b080795a66a4f05a810e042f8c28c8c"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "92228c0b7ddde1b1b3cd6fb91939d191"
server_key_iv: "e6dfbcfa3eba288f3b37fdf4d495d6f4"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.469655 13618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 08:43:24.470417 13638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.470599 13618 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:43:24.470701 13618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "5b080795a66a4f05a810e042f8c28c8c"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "92228c0b7ddde1b1b3cd6fb91939d191"
server_key_iv: "e6dfbcfa3eba288f3b37fdf4d495d6f4"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.470791 13618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:24.489354 13618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:24.489810 13618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:24.489981 13618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:24.490314 13618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:24.490697 13618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:24.490756 13618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.490797 13618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:24.490826 13618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.497915 13618 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:40823
I20260809 08:43:24.497977 13826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:40823 every 8 connection(s)
I20260809 08:43:24.498389 13618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:43:24.503770 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:24.499282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:48957 (local address 127.31.63.126:46303)
0809 08:43:24.499364 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:24.499366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:24.499543 (+ 177us) server_negotiation.cc:407] Connection header received
0809 08:43:24.500052 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:24.500053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:24.500062 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:24.500128 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:24.500853 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.501365 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.502439 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.502628 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.502689 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:24.503524 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:24.503525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:24.503526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:43:24.503530 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:24.503546 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:43:24.503550 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:24.503593 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:24.503678 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 08:43:24.503704 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 08:43:24.503988 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:24.499188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:46303 (local address 127.31.63.67:48957)
0809 08:43:24.499405 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:24.499448 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:24.499968 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:24.500232 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:24.500236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:24.500244 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:24.500776 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:24.500781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.501462 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.501463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:24.502361 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:24.502364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:24.502727 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:24.502728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:24.502855 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:24.503389 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:43:24.503390 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:24.503421 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:43:24.503445 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:24.503605 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:24.503610 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:24.503613 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:43:24.503677 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 08:43:24.503764 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260809 08:43:24.504508 13828 heartbeater.cc:347] Connected to a master server at 127.31.63.126:46303
I20260809 08:43:24.504623 13828 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:24.504827 13828 heartbeater.cc:511] Master 127.31.63.126:46303 requested a full tablet report, sending...
I20260809 08:43:24.505255 13161 ts_manager.cc:194] Registered new tserver with Master: 5b080795a66a4f05a810e042f8c28c8c (127.31.63.67:40823)
I20260809 08:43:24.505383 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 13618
I20260809 08:43:24.505481 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:43:24.505708 31997 external_mini_cluster.cc:1428] Setting key b808a62157f7cb9b99e745933313fbbb
I20260809 08:43:24.505796 13161 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.67:48957
I20260809 08:43:24.506724 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 08:43:24.508577 13239 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:24.508061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.31.63.126:46303)
0809 08:43:24.508104 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:24.508107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:24.508226 (+ 119us) server_negotiation.cc:407] Connection header received
0809 08:43:24.508305 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:24.508306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:24.508318 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:24.508367 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:24.508499 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36590: BlockingRecv error: recv got EOF from 127.0.0.1:36590 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:43:24.508616 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 13243
I20260809 08:43:24.526667 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 13426
I20260809 08:43:24.545320 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 13618
I20260809 08:43:24.563737 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 13112
2026-08-09T08:43:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (660 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T08:43:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:43:24Z Disabled control of system clock
I20260809 08:43:24.596597 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:35539
--webserver_interface=127.31.63.126
--webserver_port=0
--builtin_ntp_servers=127.31.63.84:41187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:35539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 08:43:24.682502 13842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:24.682734 13842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:24.682758 13842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:24.683964 13842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:43:24.684005 13842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:24.684020 13842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:24.684034 13842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:43:24.684047 13842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:43:24.685489 13842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:35539
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:35539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:24.685793 13842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:24.686110 13842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:24.691697 13851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:24.691717 13842 server_base.cc:1034] running on GCE node
W20260809 08:43:24.691725 13854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.691684 13849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:24.692255 13842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:24.692575 13842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:24.693739 13842 hybrid_clock.cc:648] HybridClock initialized: now 1786265004693726 us; error 32 us; skew 500 ppm
I20260809 08:43:24.695037 13842 webserver.cc:460] Webserver started at http://127.31.63.126:38097/ using document root <none> and password file <none>
I20260809 08:43:24.695336 13842 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:24.695410 13842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:24.695571 13842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:24.696709 13842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "a9d64dfeadd048cdae453360478ffcd2"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "af6b745015703846f3e4111f2d46e378"
server_key_iv: "be2ab134f198d3685ae5cc0c93ed7100"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.697082 13842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "a9d64dfeadd048cdae453360478ffcd2"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "af6b745015703846f3e4111f2d46e378"
server_key_iv: "be2ab134f198d3685ae5cc0c93ed7100"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.698566 13842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:43:24.699317 13862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.699496 13842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:43:24.699606 13842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "a9d64dfeadd048cdae453360478ffcd2"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "af6b745015703846f3e4111f2d46e378"
server_key_iv: "be2ab134f198d3685ae5cc0c93ed7100"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.699683 13842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:24.703353 13842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:24.703636 13842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:24.703785 13842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:24.707787 13842 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:35539
I20260809 08:43:24.707825 13935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:35539 every 8 connection(s)
I20260809 08:43:24.708227 13842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:43:24.708961 13938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:24.711346 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 13842
I20260809 08:43:24.711429 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:43:24.711649 31997 external_mini_cluster.cc:1428] Setting key 85415e7a3f5a126cd9ce3b35076cc952
I20260809 08:43:24.712196 13938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2: Bootstrap starting.
I20260809 08:43:24.713011 13938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:24.713366 13938 log.cc:824] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:24.714289 13938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2: No bootstrap required, opened a new log
I20260809 08:43:24.714565 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:24.712240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.31.63.126:35539)
0809 08:43:24.712440 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:24.712449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:24.712481 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:43:24.713099 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:24.713109 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:24.713177 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:24.713359 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:24.714078 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:24.714082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:24.714083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:43:24.714088 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:24.714139 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:43:24.714153 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:24.714266 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:24.714297 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 08:43:24.714357 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260809 08:43:24.716442 13938 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d64dfeadd048cdae453360478ffcd2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35539 } }
I20260809 08:43:24.716580 13938 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:24.716603 13938 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9d64dfeadd048cdae453360478ffcd2, State: Initialized, Role: FOLLOWER
I20260809 08:43:24.716733 13938 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d64dfeadd048cdae453360478ffcd2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35539 } }
I20260809 08:43:24.716810 13938 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:43:24.716833 13938 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:43:24.716861 13938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:24.717357 13938 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d64dfeadd048cdae453360478ffcd2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35539 } }
I20260809 08:43:24.717442 13938 leader_election.cc:302] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9d64dfeadd048cdae453360478ffcd2; no voters:
I20260809 08:43:24.717628 13938 leader_election.cc:288] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:43:24.717691 13946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:24.717861 13946 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [term 1 LEADER]: Becoming Leader. State: Replica: a9d64dfeadd048cdae453360478ffcd2, State: Running, Role: LEADER
I20260809 08:43:24.717965 13938 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:43:24.718020 13946 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d64dfeadd048cdae453360478ffcd2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35539 } }
I20260809 08:43:24.718544 13947 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9d64dfeadd048cdae453360478ffcd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d64dfeadd048cdae453360478ffcd2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35539 } } }
I20260809 08:43:24.718535 13948 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9d64dfeadd048cdae453360478ffcd2. Latest consensus state: current_term: 1 leader_uuid: "a9d64dfeadd048cdae453360478ffcd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9d64dfeadd048cdae453360478ffcd2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35539 } } }
I20260809 08:43:24.718664 13947 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:24.718669 13948 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:24.719259 13958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:43:24.719785 13958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:43:24.721371 13958 catalog_manager.cc:1347] Generated new cluster ID: 08e6e4e66957456bacc0812dd7d8e5fc
I20260809 08:43:24.721427 13958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:43:24.743486 13958 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:43:24.743877 13958 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:43:24.748183 13958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9d64dfeadd048cdae453360478ffcd2: Generated new TSK 0
I20260809 08:43:24.748457 13958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:43:24.754532 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35539
--builtin_ntp_servers=127.31.63.84:41187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 08:43:24.857369 13969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:24.857657 13969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:24.857687 13969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:24.858913 13969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:24.858958 13969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:24.858985 13969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:43:24.860365 13969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:24.860643 13969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:24.860971 13969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:24.867102 13979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.867154 13981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.867098 13984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:24.867414 13969 server_base.cc:1034] running on GCE node
I20260809 08:43:24.867650 13969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:24.867954 13969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:24.869120 13969 hybrid_clock.cc:648] HybridClock initialized: now 1786265004869101 us; error 38 us; skew 500 ppm
I20260809 08:43:24.870522 13969 webserver.cc:460] Webserver started at http://127.31.63.65:46527/ using document root <none> and password file <none>
I20260809 08:43:24.870792 13969 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:24.870864 13969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:24.871034 13969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:24.872171 13969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "bb47fc1ec1c541718e9b00093aebb064"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "fb3f563c7ae35b1bdb7a00a9c8deb978"
server_key_iv: "1c3e78da7946437e9ed1e8db218eeb5f"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.872606 13969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "bb47fc1ec1c541718e9b00093aebb064"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "fb3f563c7ae35b1bdb7a00a9c8deb978"
server_key_iv: "1c3e78da7946437e9ed1e8db218eeb5f"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.874192 13969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:43:24.875046 13993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.875241 13969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:43:24.875356 13969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "bb47fc1ec1c541718e9b00093aebb064"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "fb3f563c7ae35b1bdb7a00a9c8deb978"
server_key_iv: "1c3e78da7946437e9ed1e8db218eeb5f"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.875432 13969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:24.879092 13969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:24.879381 13969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:24.879525 13969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:24.879747 13969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:24.880076 13969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:24.880110 13969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.880152 13969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:24.880177 13969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.886459 13969 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:37523
I20260809 08:43:24.886541 14153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:37523 every 8 connection(s)
I20260809 08:43:24.886896 13969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:43:24.890179 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:24.887689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:42617 (local address 127.31.63.126:35539)
0809 08:43:24.887737 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:24.887739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:24.888013 (+ 274us) server_negotiation.cc:407] Connection header received
0809 08:43:24.888630 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:24.888632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:24.888639 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:24.888704 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:24.889904 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:24.889906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:24.889906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:43:24.889912 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:24.889927 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:43:24.889931 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:24.889991 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:24.890090 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 08:43:24.890107 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:43:24.890404 14156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:24.887678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:35539 (local address 127.31.63.65:42617)
0809 08:43:24.887886 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:24.887919 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:24.888501 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:24.888784 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:24.888787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:24.888795 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:24.889755 (+ 960us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:43:24.889756 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:24.889784 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:43:24.889811 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:24.889991 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:24.889997 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:24.890000 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:43:24.890094 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 08:43:24.890159 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260809 08:43:24.890892 14154 heartbeater.cc:347] Connected to a master server at 127.31.63.126:35539
I20260809 08:43:24.890983 14154 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:24.891103 14154 heartbeater.cc:511] Master 127.31.63.126:35539 requested a full tablet report, sending...
I20260809 08:43:24.891355 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 13969
I20260809 08:43:24.891458 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:43:24.891698 31997 external_mini_cluster.cc:1428] Setting key d1157c1650c97131f1502a83e2f49352
I20260809 08:43:24.891705 13883 ts_manager.cc:194] Registered new tserver with Master: bb47fc1ec1c541718e9b00093aebb064 (127.31.63.65:37523)
I20260809 08:43:24.892669 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35539
--builtin_ntp_servers=127.31.63.84:41187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 08:43:24.977671 14157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:24.977901 14157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:24.977926 14157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:24.979290 14157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:24.979411 14157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:24.979457 14157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:43:24.980880 14157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:24.981103 14157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:24.981388 14157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:24.986999 14165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.986974 14166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:24.986984 14172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:24.987283 14157 server_base.cc:1034] running on GCE node
I20260809 08:43:24.987494 14157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:24.987753 14157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:24.988932 14157 hybrid_clock.cc:648] HybridClock initialized: now 1786265004988919 us; error 47 us; skew 500 ppm
I20260809 08:43:24.990353 14157 webserver.cc:460] Webserver started at http://127.31.63.66:42361/ using document root <none> and password file <none>
I20260809 08:43:24.990674 14157 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:24.990756 14157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:24.990921 14157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:24.992079 14157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "ca3bc731cd7743369e43b2ff2efcfe49"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "64588d1b891ebfcfffed9b4ac145d946"
server_key_iv: "0fbb64b8c93f9f0b9fef6c3163485d59"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.992470 14157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "ca3bc731cd7743369e43b2ff2efcfe49"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "64588d1b891ebfcfffed9b4ac145d946"
server_key_iv: "0fbb64b8c93f9f0b9fef6c3163485d59"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.993908 14157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:43:24.994695 14181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.994874 14157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:43:24.994985 14157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "ca3bc731cd7743369e43b2ff2efcfe49"
format_stamp: "Formatted at 2026-08-09 08:43:24 on dist-test-slave-gkw7"
server_key: "64588d1b891ebfcfffed9b4ac145d946"
server_key_iv: "0fbb64b8c93f9f0b9fef6c3163485d59"
server_key_version: "encryptionkey@0"
I20260809 08:43:24.995064 14157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:24.998554 14157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:24.998835 14157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:24.998984 14157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:24.999205 14157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:24.999536 14157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:24.999593 14157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:24.999632 14157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:24.999662 14157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:25.005816 14157 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:40133
I20260809 08:43:25.005920 14339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:40133 every 8 connection(s)
I20260809 08:43:25.006246 14157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:43:25.008306 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 14157
I20260809 08:43:25.008419 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:43:25.008679 31997 external_mini_cluster.cc:1428] Setting key 4e72a731a33495e5d5c7b160eb6ff36c
I20260809 08:43:25.009507 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:25.007001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:41627 (local address 127.31.63.126:35539)
0809 08:43:25.007061 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:25.007063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:25.007281 (+ 218us) server_negotiation.cc:407] Connection header received
0809 08:43:25.007761 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:25.007762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:25.007767 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:25.007832 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:25.009222 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:25.009224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:25.009224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:43:25.009229 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:25.009248 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:43:25.009253 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:25.009315 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:25.009393 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 08:43:25.009409 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:43:25.009796 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:25.006956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:35539 (local address 127.31.63.66:41627)
0809 08:43:25.007152 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:25.007192 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:25.007661 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:25.007884 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:25.007887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:25.007896 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:25.009046 (+ 1150us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:43:25.009047 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:25.009087 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:43:25.009123 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:25.009321 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:25.009324 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:25.009326 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:43:25.009379 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:43:25.009467 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260809 08:43:25.009914 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35539
--builtin_ntp_servers=127.31.63.84:41187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 08:43:25.010277 14341 heartbeater.cc:347] Connected to a master server at 127.31.63.126:35539
I20260809 08:43:25.010373 14341 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:25.010527 14341 heartbeater.cc:511] Master 127.31.63.126:35539 requested a full tablet report, sending...
I20260809 08:43:25.010947 13883 ts_manager.cc:194] Registered new tserver with Master: ca3bc731cd7743369e43b2ff2efcfe49 (127.31.63.66:40133)
W20260809 08:43:25.097010 14346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:25.097226 14346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:25.097250 14346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:25.098475 14346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:25.098523 14346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:25.098551 14346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:43:25.099956 14346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:25.100231 14346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:25.100558 14346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:25.106124 14353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:25.106097 14357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:25.106336 14346 server_base.cc:1034] running on GCE node
W20260809 08:43:25.106099 14354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:25.106622 14346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:25.106921 14346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:25.108124 14346 hybrid_clock.cc:648] HybridClock initialized: now 1786265005108106 us; error 26 us; skew 500 ppm
I20260809 08:43:25.109810 14346 webserver.cc:460] Webserver started at http://127.31.63.67:46269/ using document root <none> and password file <none>
I20260809 08:43:25.110117 14346 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:25.110199 14346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:25.110374 14346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:25.111575 14346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "542a95fa08c7472dbcf1e948d486af11"
format_stamp: "Formatted at 2026-08-09 08:43:25 on dist-test-slave-gkw7"
server_key: "1bf60d318e88dc0cd24acfa088ecda43"
server_key_iv: "d0bcf5782a433e5a2584ca30f7616c65"
server_key_version: "encryptionkey@0"
I20260809 08:43:25.112040 14346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "542a95fa08c7472dbcf1e948d486af11"
format_stamp: "Formatted at 2026-08-09 08:43:25 on dist-test-slave-gkw7"
server_key: "1bf60d318e88dc0cd24acfa088ecda43"
server_key_iv: "d0bcf5782a433e5a2584ca30f7616c65"
server_key_version: "encryptionkey@0"
I20260809 08:43:25.113619 14346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:43:25.114517 14365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:25.114730 14346 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:25.114846 14346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "542a95fa08c7472dbcf1e948d486af11"
format_stamp: "Formatted at 2026-08-09 08:43:25 on dist-test-slave-gkw7"
server_key: "1bf60d318e88dc0cd24acfa088ecda43"
server_key_iv: "d0bcf5782a433e5a2584ca30f7616c65"
server_key_version: "encryptionkey@0"
I20260809 08:43:25.114934 14346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:25.118764 14346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:25.119065 14346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:25.119256 14346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:25.119526 14346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:25.119871 14346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:25.119927 14346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:25.119967 14346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:25.119994 14346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:25.126504 14346 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:40655
I20260809 08:43:25.126551 14523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:40655 every 8 connection(s)
I20260809 08:43:25.126902 14346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:43:25.129981 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:25.127692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:34877 (local address 127.31.63.126:35539)
0809 08:43:25.127761 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:25.127764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:25.128068 (+ 304us) server_negotiation.cc:407] Connection header received
0809 08:43:25.128532 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:25.128533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:25.128539 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:25.128599 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:25.129787 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:25.129788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:25.129789 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:43:25.129793 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:25.129808 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:43:25.129812 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:25.129872 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:25.129916 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 08:43:25.129928 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:43:25.130239 14529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:25.127692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:35539 (local address 127.31.63.67:34877)
0809 08:43:25.127936 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:25.127971 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:25.128440 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:25.128651 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:25.128655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:25.128663 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:25.129628 (+ 965us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:43:25.129629 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:25.129672 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:43:25.129702 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:25.129869 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:25.129871 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:25.129873 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:43:25.129924 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 08:43:25.129972 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":97,"threads_started":1}
I20260809 08:43:25.130798 14525 heartbeater.cc:347] Connected to a master server at 127.31.63.126:35539
I20260809 08:43:25.130904 14525 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:25.131080 14525 heartbeater.cc:511] Master 127.31.63.126:35539 requested a full tablet report, sending...
I20260809 08:43:25.131547 13883 ts_manager.cc:194] Registered new tserver with Master: 542a95fa08c7472dbcf1e948d486af11 (127.31.63.67:40655)
I20260809 08:43:25.136256 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 14346
I20260809 08:43:25.136350 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:43:25.136525 31997 external_mini_cluster.cc:1428] Setting key 31dc271ba4a2f626f860e58aa2c6f069
I20260809 08:43:25.137647 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 08:43:25.139421 13943 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:43:25.139003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47666 (local address 127.31.63.126:35539)
0809 08:43:25.139063 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:25.139065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:25.139163 (+ 98us) server_negotiation.cc:407] Connection header received
0809 08:43:25.139214 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:25.139214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:25.139219 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:25.139276 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:25.139373 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47666: BlockingRecv error: recv got EOF from 127.0.0.1:47666 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:43:25.139504 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 13969
I20260809 08:43:25.166567 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 14157
I20260809 08:43:25.179551 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 14346
I20260809 08:43:25.193794 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 13842
2026-08-09T08:43:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (632 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:43:25 dist-test-slave-gkw7 krb5kdc[14542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:43:25 dist-test-slave-gkw7 krb5kdc[14542](info): set up 2 sockets
Aug 09 08:43:25 dist-test-slave-gkw7 krb5kdc[14542](info): commencing operation
krb5kdc: starting...
W20260809 08:43:26.442700 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:43:26 dist-test-slave-gkw7 krb5kdc[14542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:43:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:43:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:43:26.573024 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37663
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:37663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:26.656574 14562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:26.656834 14562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:26.656873 14562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:26.658102 14562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:43:26.658238 14562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:26.658262 14562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:26.658284 14562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:43:26.658306 14562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:43:26.659870 14562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:37663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:26.660163 14562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:26.660405 14562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:26.665622 14570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:26.665632 14572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:26.665679 14569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:26.665820 14562 server_base.cc:1034] running on GCE node
I20260809 08:43:26.666217 14562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:26.666525 14562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:26.667693 14562 hybrid_clock.cc:648] HybridClock initialized: now 1786265006667686 us; error 38 us; skew 500 ppm
Aug 09 08:43:26 dist-test-slave-gkw7 krb5kdc[14542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:26.670382 14562 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:43:26.670850 14562 webserver.cc:460] Webserver started at http://127.31.63.126:45967/ using document root <none> and password file <none>
I20260809 08:43:26.671133 14562 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:26.671211 14562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:26.671363 14562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:26.672488 14562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "fa7b4ec32cea45ecb0bb3ad72839279d"
format_stamp: "Formatted at 2026-08-09 08:43:26 on dist-test-slave-gkw7"
server_key: "753f70803fcf1b7ad3a53b5674fd8e5c"
server_key_iv: "78fa10240a703eb8c70bc605081d8288"
server_key_version: "encryptionkey@0"
I20260809 08:43:26.672869 14562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "fa7b4ec32cea45ecb0bb3ad72839279d"
format_stamp: "Formatted at 2026-08-09 08:43:26 on dist-test-slave-gkw7"
server_key: "753f70803fcf1b7ad3a53b5674fd8e5c"
server_key_iv: "78fa10240a703eb8c70bc605081d8288"
server_key_version: "encryptionkey@0"
I20260809 08:43:26.674343 14562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:43:26.675108 14585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:26.675288 14562 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:43:26.675385 14562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "fa7b4ec32cea45ecb0bb3ad72839279d"
format_stamp: "Formatted at 2026-08-09 08:43:26 on dist-test-slave-gkw7"
server_key: "753f70803fcf1b7ad3a53b5674fd8e5c"
server_key_iv: "78fa10240a703eb8c70bc605081d8288"
server_key_version: "encryptionkey@0"
I20260809 08:43:26.675469 14562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:26.697468 14562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:26.699705 14562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:26.699883 14562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:26.704169 14562 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:37663
I20260809 08:43:26.704226 14660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:37663 every 8 connection(s)
I20260809 08:43:26.704627 14562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:43:26.705746 14663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:26.708146 14663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d: Bootstrap starting.
I20260809 08:43:26.708823 14663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:26.708912 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 14562
I20260809 08:43:26.709029 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:43:26.709194 14663 log.cc:824] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:26.709252 31997 external_mini_cluster.cc:1428] Setting key 5f155aaa15e53150f98f117c5ed7a476
I20260809 08:43:26.710130 14663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d: No bootstrap required, opened a new log
I20260809 08:43:26.712203 14663 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b4ec32cea45ecb0bb3ad72839279d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37663 } }
I20260809 08:43:26.712348 14663 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:26.712371 14663 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa7b4ec32cea45ecb0bb3ad72839279d, State: Initialized, Role: FOLLOWER
I20260809 08:43:26.712503 14663 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b4ec32cea45ecb0bb3ad72839279d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37663 } }
I20260809 08:43:26.712596 14663 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:43:26.712621 14663 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:43:26.712689 14663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:26.713227 14663 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b4ec32cea45ecb0bb3ad72839279d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37663 } }
I20260809 08:43:26.713361 14663 leader_election.cc:302] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa7b4ec32cea45ecb0bb3ad72839279d; no voters:
I20260809 08:43:26.713581 14663 leader_election.cc:288] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:43:26.713599 14668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:26.713729 14668 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [term 1 LEADER]: Becoming Leader. State: Replica: fa7b4ec32cea45ecb0bb3ad72839279d, State: Running, Role: LEADER
I20260809 08:43:26.713821 14668 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b4ec32cea45ecb0bb3ad72839279d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37663 } }
I20260809 08:43:26.713869 14663 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:43:26.714156 14668 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa7b4ec32cea45ecb0bb3ad72839279d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b4ec32cea45ecb0bb3ad72839279d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37663 } } }
I20260809 08:43:26.714228 14668 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:26.714367 14668 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa7b4ec32cea45ecb0bb3ad72839279d. Latest consensus state: current_term: 1 leader_uuid: "fa7b4ec32cea45ecb0bb3ad72839279d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa7b4ec32cea45ecb0bb3ad72839279d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37663 } } }
I20260809 08:43:26.714438 14668 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d [sys.catalog]: This master's current role is: LEADER
I20260809 08:43:26.714716 14679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:43:26.715247 14679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:43:26 dist-test-slave-gkw7 krb5kdc[14542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:43:26.717144 14679 catalog_manager.cc:1347] Generated new cluster ID: cad297f463ad4fff807a326ca8f3523f
I20260809 08:43:26.717195 14679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:43:26.721108 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:26.709804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53568 (local address 127.31.63.126:37663)
0809 08:43:26.709979 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:26.709989 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:26.710028 (+ 39us) server_negotiation.cc:407] Connection header received
0809 08:43:26.710587 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:26.710596 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:26.710650 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:26.710801 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:26.711443 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:26.712333 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:26.713246 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:26.714174 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:26.714638 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:26.717380 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:26.717383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:26.717384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:26.717395 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:26.719324 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:26.719713 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:26.719716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:26.719717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:26.719811 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:26.719975 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:26.719975 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:26.719976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:26.720242 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:26.720380 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:26.720775 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 08:43:26.720908 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260809 08:43:26.724014 14679 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:43:26.724480 14679 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:43:26.728348 14679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d: Generated new TSK 0
I20260809 08:43:26.728518 14679 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:43:26.758458 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37663
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:26.844054 14701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:26.844307 14701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:26.844347 14701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:26.845521 14701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:26.845613 14701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:26.845661 14701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:43:26.847215 14701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:26.847481 14701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:26.847723 14701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:26.852871 14715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:26.852913 14701 server_base.cc:1034] running on GCE node
W20260809 08:43:26.852880 14712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:26.852876 14711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:26.853350 14701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:26.853670 14701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:26.854820 14701 hybrid_clock.cc:648] HybridClock initialized: now 1786265006854807 us; error 51 us; skew 500 ppm
Aug 09 08:43:26 dist-test-slave-gkw7 krb5kdc[14542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:26.857365 14701 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:43:26.857887 14701 webserver.cc:460] Webserver started at http://127.31.63.65:39451/ using document root <none> and password file <none>
I20260809 08:43:26.858119 14701 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:26.858186 14701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:26.858355 14701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:26.859406 14701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "63e8ee4d69ef4dae82ea5022aa0db37e"
format_stamp: "Formatted at 2026-08-09 08:43:26 on dist-test-slave-gkw7"
server_key: "5bc993fe12fd5ff017ae6e31bc83252a"
server_key_iv: "a93e5eb29597cb46b94f0b2b25bf657a"
server_key_version: "encryptionkey@0"
I20260809 08:43:26.859784 14701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "63e8ee4d69ef4dae82ea5022aa0db37e"
format_stamp: "Formatted at 2026-08-09 08:43:26 on dist-test-slave-gkw7"
server_key: "5bc993fe12fd5ff017ae6e31bc83252a"
server_key_iv: "a93e5eb29597cb46b94f0b2b25bf657a"
server_key_version: "encryptionkey@0"
I20260809 08:43:26.861101 14701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:43:26.861958 14734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:26.862222 14701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:43:26.862362 14701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "63e8ee4d69ef4dae82ea5022aa0db37e"
format_stamp: "Formatted at 2026-08-09 08:43:26 on dist-test-slave-gkw7"
server_key: "5bc993fe12fd5ff017ae6e31bc83252a"
server_key_iv: "a93e5eb29597cb46b94f0b2b25bf657a"
server_key_version: "encryptionkey@0"
I20260809 08:43:26.862461 14701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:26.887385 14701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:26.889603 14701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:26.889810 14701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:26.890105 14701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:26.890496 14701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:26.890560 14701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:26.890601 14701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:26.890636 14701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:26.897651 14701 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:36403
I20260809 08:43:26.897739 14889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:36403 every 8 connection(s)
I20260809 08:43:26.898132 14701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:43:26 dist-test-slave-gkw7 krb5kdc[14542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265006, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:43:26.904831 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 14701
I20260809 08:43:26.905063 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:43:26.905333 31997 external_mini_cluster.cc:1428] Setting key 71e3b9d438d775da3d84441b96a90f00
I20260809 08:43:26.908133 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:26.898952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:36035 (local address 127.31.63.126:37663)
0809 08:43:26.899032 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:26.899034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:26.899262 (+ 228us) server_negotiation.cc:407] Connection header received
0809 08:43:26.899761 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:26.899763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:26.899773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:26.899841 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:26.900774 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:26.901330 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:26.902349 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:26.902556 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:26.902624 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:26.904996 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:26.904998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:26.904999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:26.905007 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:26.906757 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:26.907138 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:26.907140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:26.907140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:26.907205 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:26.907441 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:26.907442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:26.907442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:26.907590 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:26.907661 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:26.907908 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 08:43:26.907998 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 08:43:26.908280 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:26.898921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37663 (local address 127.31.63.65:36035)
0809 08:43:26.899105 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:26.899151 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:26.899667 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:26.899896 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:26.899899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:26.900480 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:26.900689 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:26.900694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:26.901417 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:26.901418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:26.902247 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:26.902248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:26.902639 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:26.902640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:26.902752 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:26.903315 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:26.903323 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:26.904802 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:26.906841 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:26.906848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:26.906850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:26.907062 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:26.907274 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:26.907274 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:26.907275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:26.907375 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:26.907655 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:26.907659 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:26.907781 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 08:43:26.907862 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 08:43:26.908009 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260809 08:43:26.908950 14890 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37663
I20260809 08:43:26.909065 14890 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:26.909252 14890 heartbeater.cc:511] Master 127.31.63.126:37663 requested a full tablet report, sending...
I20260809 08:43:26.909924 14615 ts_manager.cc:194] Registered new tserver with Master: 63e8ee4d69ef4dae82ea5022aa0db37e (127.31.63.65:36403)
I20260809 08:43:26.910883 14615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:36035
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:43:26.933118 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37663
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:27.018419 14901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:27.018641 14901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:27.018667 14901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:27.019865 14901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:27.019908 14901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:27.019935 14901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:43:27.021442 14901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:27.021720 14901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:27.021992 14901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:27.027315 14913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:27.027364 14910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:27.027324 14909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:27.027558 14901 server_base.cc:1034] running on GCE node
I20260809 08:43:27.027817 14901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:27.028107 14901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:27.029316 14901 hybrid_clock.cc:648] HybridClock initialized: now 1786265007029293 us; error 37 us; skew 500 ppm
Aug 09 08:43:27 dist-test-slave-gkw7 krb5kdc[14542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:27.031960 14901 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:43:27.032459 14901 webserver.cc:460] Webserver started at http://127.31.63.66:36457/ using document root <none> and password file <none>
I20260809 08:43:27.032709 14901 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:27.032757 14901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:27.032898 14901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:27.034068 14901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "1f5b364af6c449598baae1d533432df8"
format_stamp: "Formatted at 2026-08-09 08:43:27 on dist-test-slave-gkw7"
server_key: "88c0ca3dbf10b2d7bba98fbac2d6efcb"
server_key_iv: "3e2b97388f904f62d4addf872d526560"
server_key_version: "encryptionkey@0"
I20260809 08:43:27.034442 14901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "1f5b364af6c449598baae1d533432df8"
format_stamp: "Formatted at 2026-08-09 08:43:27 on dist-test-slave-gkw7"
server_key: "88c0ca3dbf10b2d7bba98fbac2d6efcb"
server_key_iv: "3e2b97388f904f62d4addf872d526560"
server_key_version: "encryptionkey@0"
I20260809 08:43:27.035823 14901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 08:43:27.036639 14925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:27.036835 14901 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:27.036948 14901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "1f5b364af6c449598baae1d533432df8"
format_stamp: "Formatted at 2026-08-09 08:43:27 on dist-test-slave-gkw7"
server_key: "88c0ca3dbf10b2d7bba98fbac2d6efcb"
server_key_iv: "3e2b97388f904f62d4addf872d526560"
server_key_version: "encryptionkey@0"
I20260809 08:43:27.037026 14901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:27.052131 14901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:27.054543 14901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:27.054754 14901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:27.055053 14901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:27.055464 14901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:27.055533 14901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:27.055585 14901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:27.055610 14901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:27.062460 14901 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:37033
I20260809 08:43:27.062515 15092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:37033 every 8 connection(s)
I20260809 08:43:27.062928 14901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:43:27.068872 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 14901
I20260809 08:43:27.068971 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:43:27.069216 31997 external_mini_cluster.cc:1428] Setting key a2eae017953a98fd9183a590e8fcc5e1
Aug 09 08:43:27 dist-test-slave-gkw7 krb5kdc[14542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:43:27.073683 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.063775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:47341 (local address 127.31.63.126:37663)
0809 08:43:27.063853 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:27.063856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:27.064160 (+ 304us) server_negotiation.cc:407] Connection header received
0809 08:43:27.064871 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:27.064872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:27.064884 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:27.064940 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:27.066029 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.066539 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.067634 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.067830 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.067886 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.070480 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:27.070482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:27.070482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:27.070491 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:27.072056 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:27.072518 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:27.072519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:27.072520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:27.072596 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:27.072849 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:27.072850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:27.072851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:27.073012 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:27.073107 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:27.073390 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 08:43:27.073499 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:43:27.073905 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.063745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37663 (local address 127.31.63.66:47341)
0809 08:43:27.063973 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:27.064029 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:27.064772 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:27.065023 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:27.065028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:27.065727 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:27.065945 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.065951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.066696 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.066697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.067552 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.067554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.067934 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.067935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.068056 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.068564 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:27.068574 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:27.070373 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:27.072191 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:27.072198 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:27.072200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:27.072438 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:27.072668 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:27.072668 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:27.072669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:27.072782 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:27.073108 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:27.073115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:27.073243 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 08:43:27.073337 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 08:43:27.073468 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":111,"threads_started":1}
I20260809 08:43:27.074429 15094 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37663
I20260809 08:43:27.074546 15094 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:27.074785 15094 heartbeater.cc:511] Master 127.31.63.126:37663 requested a full tablet report, sending...
I20260809 08:43:27.075309 14615 ts_manager.cc:194] Registered new tserver with Master: 1f5b364af6c449598baae1d533432df8 (127.31.63.66:37033)
I20260809 08:43:27.075703 14615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:47341
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:43:27.101116 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37663
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:43:27.191699 15100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:43:27.191901 15100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:43:27.191924 15100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:43:27.192996 15100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:43:27.193030 15100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:43:27.193063 15100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:43:27.194478 15100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:43:27.194658 15100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:43:27.194828 15100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:43:27.199839 15106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:27.199847 15105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:43:27.199847 15111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:43:27.200003 15100 server_base.cc:1034] running on GCE node
I20260809 08:43:27.200346 15100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:43:27.200606 15100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:43:27.201814 15100 hybrid_clock.cc:648] HybridClock initialized: now 1786265007201791 us; error 39 us; skew 500 ppm
Aug 09 08:43:27 dist-test-slave-gkw7 krb5kdc[14542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:43:27.204520 15100 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:43:27.204984 15100 webserver.cc:460] Webserver started at http://127.31.63.67:45257/ using document root <none> and password file <none>
I20260809 08:43:27.205220 15100 fs_manager.cc:359] Metadata directory not provided
I20260809 08:43:27.205292 15100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:43:27.205454 15100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:43:27.206573 15100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "8eceb9371b9843f895deb19fc5c3e6d1"
format_stamp: "Formatted at 2026-08-09 08:43:27 on dist-test-slave-gkw7"
server_key: "dd77f4ba4f6b9928ab904bd1356d41df"
server_key_iv: "db2c2668e3b1e26b2a28375779de959c"
server_key_version: "encryptionkey@0"
I20260809 08:43:27.207041 15100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "8eceb9371b9843f895deb19fc5c3e6d1"
format_stamp: "Formatted at 2026-08-09 08:43:27 on dist-test-slave-gkw7"
server_key: "dd77f4ba4f6b9928ab904bd1356d41df"
server_key_iv: "db2c2668e3b1e26b2a28375779de959c"
server_key_version: "encryptionkey@0"
I20260809 08:43:27.208473 15100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:43:27.209339 15123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:27.209589 15100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:43:27.209702 15100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "8eceb9371b9843f895deb19fc5c3e6d1"
format_stamp: "Formatted at 2026-08-09 08:43:27 on dist-test-slave-gkw7"
server_key: "dd77f4ba4f6b9928ab904bd1356d41df"
server_key_iv: "db2c2668e3b1e26b2a28375779de959c"
server_key_version: "encryptionkey@0"
I20260809 08:43:27.209793 15100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:43:27.218021 15100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:43:27.219985 15100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:43:27.220155 15100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:43:27.220387 15100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:43:27.220713 15100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:43:27.220769 15100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:27.220811 15100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:43:27.220839 15100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:43:27.227744 15100 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:33461
I20260809 08:43:27.227828 15279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:33461 every 8 connection(s)
I20260809 08:43:27.228165 15100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:43:27 dist-test-slave-gkw7 krb5kdc[14542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265007, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:43:27.237506 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 15100
I20260809 08:43:27.237640 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:43:27.237777 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.229060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:54339 (local address 127.31.63.126:37663)
0809 08:43:27.229130 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:27.229133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:27.229369 (+ 236us) server_negotiation.cc:407] Connection header received
0809 08:43:27.229941 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:27.229942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:27.229953 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:27.230019 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:27.230946 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.231448 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.232435 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.232621 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.232679 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.234879 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:27.234881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:27.234881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:27.234888 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:27.236282 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:27.236762 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:27.236763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:27.236763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:27.236817 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:27.237034 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:27.237035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:27.237035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:27.237136 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:27.237184 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:27.237525 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 08:43:27.237668 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:43:27.237859 31997 external_mini_cluster.cc:1428] Setting key f75dde906541b30281ba61fb1f476bf5
I20260809 08:43:27.237977 15283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.229046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37663 (local address 127.31.63.67:54339)
0809 08:43:27.229233 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:27.229276 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:27.229852 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:27.230082 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:27.230086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:27.230670 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:43:27.230869 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.230873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.231531 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.231532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.232353 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.232355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.232703 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.232704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.232825 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.233304 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:43:27.233312 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:43:27.234785 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:43:27.236460 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:27.236467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:27.236469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:27.236688 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:27.236880 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:43:27.236880 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:43:27.236881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:43:27.236976 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:43:27.237190 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:43:27.237195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:43:27.237377 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 08:43:27.237504 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 08:43:27.237728 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260809 08:43:27.238615 15280 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37663
I20260809 08:43:27.238729 15280 heartbeater.cc:464] Registering TS with master...
I20260809 08:43:27.238955 15280 heartbeater.cc:511] Master 127.31.63.126:37663 requested a full tablet report, sending...
I20260809 08:43:27.239351 14615 ts_manager.cc:194] Registered new tserver with Master: 8eceb9371b9843f895deb19fc5c3e6d1 (127.31.63.67:33461)
I20260809 08:43:27.239724 14615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:54339
I20260809 08:43:27.249993 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:43:27.257620 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.251367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53582 (local address 127.31.63.126:37663)
0809 08:43:27.251431 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:27.251432 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:27.251470 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:43:27.251534 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:27.251535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:27.251544 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:27.251592 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:43:27.252143 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.252659 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.253354 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.253583 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.253645 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.254781 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:43:27.254782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:43:27.254782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:43:27.254788 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:43:27.256489 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:27.256821 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:27.256822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:27.256822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:27.256875 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:43:27.257008 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:43:27.257008 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:43:27.257008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:43:27.257149 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:43:27.257268 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:27.257386 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 08:43:27.257448 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:43:27.259116 14615 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:43:27.260726 14615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:43:27.267817 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.263167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:36403 (local address 127.0.0.1:55440)
0809 08:43:27.263717 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:27.263727 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:27.263787 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:27.264082 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:27.264082 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:27.264084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:27.264140 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.264141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.265440 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.265441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.266515 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.266518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.267487 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.267489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.267576 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.267590 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:43:27.267664 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 08:43:27.267718 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":98,"thread_start_us":108,"threads_started":1}
I20260809 08:43:27.267836 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.263311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55440 (local address 127.31.63.65:36403)
0809 08:43:27.263561 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:27.263565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:27.263785 (+ 220us) server_negotiation.cc:407] Connection header received
0809 08:43:27.263902 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:27.263905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:27.263922 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:27.263996 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:27.264223 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.265315 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.266617 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.267391 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.267467 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.267525 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:27.267696 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 08:43:27.267753 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":102,"threads_started":1}
I20260809 08:43:27.268007 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.263524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44974 (local address 127.31.63.67:33461)
0809 08:43:27.263735 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:27.263739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:27.264305 (+ 566us) server_negotiation.cc:407] Connection header received
0809 08:43:27.264366 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:27.264369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:27.264383 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:27.264474 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:27.264579 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.265406 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.266962 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.267499 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.267568 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.267625 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:27.267862 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 08:43:27.267907 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":100,"threads_started":1}
I20260809 08:43:27.268126 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.263311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.31.63.66:37033)
0809 08:43:27.263669 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:27.263673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:27.263717 (+ 44us) server_negotiation.cc:407] Connection header received
0809 08:43:27.263763 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:27.263765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:27.263782 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:27.263877 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:27.264064 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.265278 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.266935 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.267697 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.267766 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.267818 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:27.268008 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 08:43:27.268050 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":105,"threads_started":1}
I20260809 08:43:27.268100 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.263167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:37033 (local address 127.0.0.1:59832)
0809 08:43:27.263533 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:27.263580 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:27.263650 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:27.263899 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:27.263902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:27.263905 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:27.263975 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.263977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.265387 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.265388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.266858 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.266860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.267858 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.267859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.267930 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.267934 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:43:27.267991 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 08:43:27.268032 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":41,"thread_start_us":69,"threads_started":1}
I20260809 08:43:27.268424 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.263381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:33461 (local address 127.0.0.1:44974)
0809 08:43:27.264229 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:27.264237 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:27.264298 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:27.264471 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:27.264471 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:27.264473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:27.264512 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.264513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.265525 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.265526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.266857 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.266860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.267659 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.267660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.267712 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.267716 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:43:27.267762 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 08:43:27.267795 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"mutex_wait_us":8,"thread_start_us":77,"threads_started":1}
I20260809 08:43:27.269098 14996 tablet_service.cc:1467] Processing CreateTablet for tablet e18a5beab9b54431a46eb4fd84b02aae (DEFAULT_TABLE table=test-table [id=ed007c64596a408a9b15227819a2013c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:27.269106 14796 tablet_service.cc:1467] Processing CreateTablet for tablet e18a5beab9b54431a46eb4fd84b02aae (DEFAULT_TABLE table=test-table [id=ed007c64596a408a9b15227819a2013c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:27.269188 15190 tablet_service.cc:1467] Processing CreateTablet for tablet e18a5beab9b54431a46eb4fd84b02aae (DEFAULT_TABLE table=test-table [id=ed007c64596a408a9b15227819a2013c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:43:27.269469 14996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18a5beab9b54431a46eb4fd84b02aae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:27.269488 14796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18a5beab9b54431a46eb4fd84b02aae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:27.269487 15190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18a5beab9b54431a46eb4fd84b02aae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:43:27.272150 15303 tablet_bootstrap.cc:492] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: Bootstrap starting.
I20260809 08:43:27.273252 15303 tablet_bootstrap.cc:654] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:27.273785 15307 tablet_bootstrap.cc:492] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: Bootstrap starting.
I20260809 08:43:27.273931 15303 log.cc:824] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:27.274573 15307 tablet_bootstrap.cc:654] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:27.274647 15306 tablet_bootstrap.cc:492] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e: Bootstrap starting.
I20260809 08:43:27.275239 15303 tablet_bootstrap.cc:492] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: No bootstrap required, opened a new log
I20260809 08:43:27.275178 15307 log.cc:824] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:27.275368 15306 tablet_bootstrap.cc:654] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e: Neither blocks nor log segments found. Creating new log.
I20260809 08:43:27.275425 15303 ts_tablet_manager.cc:1397] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:43:27.275970 15306 log.cc:824] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:43:27.276721 15307 tablet_bootstrap.cc:492] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: No bootstrap required, opened a new log
I20260809 08:43:27.276790 15307 ts_tablet_manager.cc:1397] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:43:27.277642 15303 raft_consensus.cc:348] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5b364af6c449598baae1d533432df8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37033 } } peers { permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 } } peers { permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 } }
I20260809 08:43:27.277832 15303 raft_consensus.cc:374] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:27.277872 15303 raft_consensus.cc:729] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5b364af6c449598baae1d533432df8, State: Initialized, Role: FOLLOWER
I20260809 08:43:27.278738 15307 raft_consensus.cc:348] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5b364af6c449598baae1d533432df8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37033 } } peers { permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 } } peers { permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 } }
I20260809 08:43:27.278903 15307 raft_consensus.cc:374] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:27.278931 15307 raft_consensus.cc:729] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eceb9371b9843f895deb19fc5c3e6d1, State: Initialized, Role: FOLLOWER
I20260809 08:43:27.279094 15307 consensus_queue.cc:260] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5b364af6c449598baae1d533432df8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37033 } } peers { permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 } } peers { permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 } }
I20260809 08:43:27.279361 15307 ts_tablet_manager.cc:1428] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:43:27.279495 15280 heartbeater.cc:503] Master 127.31.63.126:37663 was elected leader, sending a full tablet report...
I20260809 08:43:27.282842 15306 tablet_bootstrap.cc:492] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e: No bootstrap required, opened a new log
I20260809 08:43:27.282905 15306 ts_tablet_manager.cc:1397] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260809 08:43:27.284808 15306 raft_consensus.cc:348] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5b364af6c449598baae1d533432df8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37033 } } peers { permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 } } peers { permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 } }
I20260809 08:43:27.284978 15306 raft_consensus.cc:374] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:43:27.285012 15306 raft_consensus.cc:729] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e8ee4d69ef4dae82ea5022aa0db37e, State: Initialized, Role: FOLLOWER
I20260809 08:43:27.285120 15306 consensus_queue.cc:260] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5b364af6c449598baae1d533432df8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37033 } } peers { permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 } } peers { permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 } }
I20260809 08:43:27.285411 15306 ts_tablet_manager.cc:1428] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:43:27.285804 14890 heartbeater.cc:503] Master 127.31.63.126:37663 was elected leader, sending a full tablet report...
I20260809 08:43:27.286231 15303 consensus_queue.cc:260] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5b364af6c449598baae1d533432df8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37033 } } peers { permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 } } peers { permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 } }
I20260809 08:43:27.286588 15303 ts_tablet_manager.cc:1428] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260809 08:43:27.286610 15094 heartbeater.cc:503] Master 127.31.63.126:37663 was elected leader, sending a full tablet report...
W20260809 08:43:27.313715 15095 tablet.cc:2369] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:43:27.399004 14891 tablet.cc:2369] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:43:27.404536 15321 raft_consensus.cc:482] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:43:27.404680 15321 raft_consensus.cc:504] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5b364af6c449598baae1d533432df8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37033 } } peers { permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 } } peers { permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 } }
I20260809 08:43:27.405181 15321 leader_election.cc:288] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63e8ee4d69ef4dae82ea5022aa0db37e (127.31.63.65:36403), 8eceb9371b9843f895deb19fc5c3e6d1 (127.31.63.67:33461)
I20260809 08:43:27.409510 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.405440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:36779 (local address 127.31.63.67:33461)
0809 08:43:27.405503 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:27.405506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:27.405519 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:43:27.405618 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:27.405619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:27.405628 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:27.405722 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:27.405954 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.406806 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.408424 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.409099 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.409189 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.409199 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:27.409387 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 08:43:27.409435 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:43:27.409511 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.405357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:33461 (local address 127.31.63.66:36779)
0809 08:43:27.405461 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:27.405475 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:27.405545 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:27.405742 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:27.405742 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:27.405744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:27.405846 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.405848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.406930 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.406930 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.408296 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.408299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.409196 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.409197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.409289 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.409294 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:43:27.409356 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:43:27.409412 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":91}
I20260809 08:43:27.409510 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.405476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:46133 (local address 127.31.63.65:36403)
0809 08:43:27.405547 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:27.405549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:27.405698 (+ 149us) server_negotiation.cc:407] Connection header received
0809 08:43:27.405731 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:27.405732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:27.405742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:27.405799 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:43:27.405968 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.406813 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.408423 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.409095 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.409189 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.409199 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:27.409387 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 08:43:27.409434 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:43:27.409552 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.405359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:36403 (local address 127.31.63.66:46133)
0809 08:43:27.405598 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:27.405607 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:43:27.405661 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:43:27.405823 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:43:27.405824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:43:27.405826 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:43:27.405891 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.405893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.406930 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.406930 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.408296 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:43:27.408299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.409196 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.409197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:43:27.409289 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.409294 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:43:27.409356 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:43:27.409411 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":97,"thread_start_us":122,"threads_started":1}
I20260809 08:43:27.409888 14826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18a5beab9b54431a46eb4fd84b02aae" candidate_uuid: "1f5b364af6c449598baae1d533432df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" is_pre_election: true
I20260809 08:43:27.409910 15218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18a5beab9b54431a46eb4fd84b02aae" candidate_uuid: "1f5b364af6c449598baae1d533432df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" is_pre_election: true
I20260809 08:43:27.410080 14826 raft_consensus.cc:2454] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f5b364af6c449598baae1d533432df8 in term 0.
I20260809 08:43:27.410118 15218 raft_consensus.cc:2454] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f5b364af6c449598baae1d533432df8 in term 0.
I20260809 08:43:27.410352 14930 leader_election.cc:302] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5b364af6c449598baae1d533432df8, 63e8ee4d69ef4dae82ea5022aa0db37e; no voters:
I20260809 08:43:27.410566 15321 raft_consensus.cc:2781] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:43:27.410632 15321 raft_consensus.cc:482] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:43:27.410650 15321 raft_consensus.cc:3037] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:27.411494 15321 raft_consensus.cc:504] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5b364af6c449598baae1d533432df8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37033 } } peers { permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 } } peers { permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 } }
I20260809 08:43:27.411651 15321 leader_election.cc:288] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [CANDIDATE]: Term 1 election: Requested vote from peers 63e8ee4d69ef4dae82ea5022aa0db37e (127.31.63.65:36403), 8eceb9371b9843f895deb19fc5c3e6d1 (127.31.63.67:33461)
I20260809 08:43:27.411835 14826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18a5beab9b54431a46eb4fd84b02aae" candidate_uuid: "1f5b364af6c449598baae1d533432df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e"
I20260809 08:43:27.411862 15218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18a5beab9b54431a46eb4fd84b02aae" candidate_uuid: "1f5b364af6c449598baae1d533432df8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eceb9371b9843f895deb19fc5c3e6d1"
I20260809 08:43:27.411943 14826 raft_consensus.cc:3037] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:27.411966 15218 raft_consensus.cc:3037] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:43:27.412827 15218 raft_consensus.cc:2454] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f5b364af6c449598baae1d533432df8 in term 1.
I20260809 08:43:27.412875 14826 raft_consensus.cc:2454] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f5b364af6c449598baae1d533432df8 in term 1.
I20260809 08:43:27.413111 14930 leader_election.cc:302] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5b364af6c449598baae1d533432df8, 63e8ee4d69ef4dae82ea5022aa0db37e; no voters:
I20260809 08:43:27.413233 15321 raft_consensus.cc:2781] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:43:27.413436 15321 raft_consensus.cc:686] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 1 LEADER]: Becoming Leader. State: Replica: 1f5b364af6c449598baae1d533432df8, State: Running, Role: LEADER
I20260809 08:43:27.413594 15321 consensus_queue.cc:237] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5b364af6c449598baae1d533432df8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37033 } } peers { permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 } } peers { permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 } }
I20260809 08:43:27.414547 14615 catalog_manager.cc:5495] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f5b364af6c449598baae1d533432df8 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "1f5b364af6c449598baae1d533432df8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5b364af6c449598baae1d533432df8" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 37033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 } health_report { overall_health: UNKNOWN } } }
I20260809 08:43:27.427438 15299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.425030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59842 (local address 127.31.63.66:37033)
0809 08:43:27.425104 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:43:27.425106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:43:27.425115 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:43:27.425148 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:43:27.425150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:43:27.425167 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:43:27.425234 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:43:27.425383 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.425916 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.426641 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:43:27.426846 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:43:27.426906 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:43:27.427043 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:43:27.427161 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:43:27.427232 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:43:27.427276 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 08:43:27.427297 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 08:43:27.429441 14826 raft_consensus.cc:1261] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e [term 1 FOLLOWER]: Refusing update from remote peer 1f5b364af6c449598baae1d533432df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:27.429441 15218 raft_consensus.cc:1261] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1 [term 1 FOLLOWER]: Refusing update from remote peer 1f5b364af6c449598baae1d533432df8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:43:27.429890 15321 consensus_queue.cc:1035] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63e8ee4d69ef4dae82ea5022aa0db37e" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 36403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:27.430131 15324 consensus_queue.cc:1035] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eceb9371b9843f895deb19fc5c3e6d1" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:43:27.432595 15329 mvcc.cc:205] Tried to move back new op lower bound from 7316541470427623424 to 7316541470366756864. Current Snapshot: MvccSnapshot[applied={T|T < 7316541470427623424}]
I20260809 08:43:27.433030 15330 mvcc.cc:205] Tried to move back new op lower bound from 7316541470427623424 to 7316541470366756864. Current Snapshot: MvccSnapshot[applied={T|T < 7316541470427623424}]
I20260809 08:43:27.435051 14614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:43:27.435143 14614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:43:27.435949 14614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa7b4ec32cea45ecb0bb3ad72839279d: Sending DeleteTablet for 3 replicas of tablet e18a5beab9b54431a46eb4fd84b02aae
I20260809 08:43:27.436357 14796 tablet_service.cc:1514] Processing DeleteTablet for tablet e18a5beab9b54431a46eb4fd84b02aae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:27 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:55440
I20260809 08:43:27.436376 15190 tablet_service.cc:1514] Processing DeleteTablet for tablet e18a5beab9b54431a46eb4fd84b02aae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:27 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:44974
I20260809 08:43:27.436380 14996 tablet_service.cc:1514] Processing DeleteTablet for tablet e18a5beab9b54431a46eb4fd84b02aae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:43:27 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:59832
I20260809 08:43:27.436631 15342 tablet_replica.cc:330] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e: stopping tablet replica
I20260809 08:43:27.436708 15343 tablet_replica.cc:330] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: stopping tablet replica
I20260809 08:43:27.436746 15342 raft_consensus.cc:2229] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:43:27.436810 15343 raft_consensus.cc:2229] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:43:27.436873 15342 raft_consensus.cc:2258] T e18a5beab9b54431a46eb4fd84b02aae P 63e8ee4d69ef4dae82ea5022aa0db37e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:27.436934 15343 raft_consensus.cc:2258] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:27.437022 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 14701
I20260809 08:43:27.437058 15341 tablet_replica.cc:330] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: stopping tablet replica
I20260809 08:43:27.442035 15341 raft_consensus.cc:2229] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:43:27.442214 15341 raft_consensus.cc:2258] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:43:27.443151 15343 ts_tablet_manager.cc:1905] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:27.443877 15341 ts_tablet_manager.cc:1905] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:43:27.445072 15343 ts_tablet_manager.cc:1918] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:43:27.445174 15343 log.cc:1196] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/e18a5beab9b54431a46eb4fd84b02aae
I20260809 08:43:27.445503 15343 ts_tablet_manager.cc:1939] T e18a5beab9b54431a46eb4fd84b02aae P 1f5b364af6c449598baae1d533432df8: Deleting consensus metadata
I20260809 08:43:27.446079 15341 ts_tablet_manager.cc:1918] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:43:27.446183 15341 log.cc:1196] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/e18a5beab9b54431a46eb4fd84b02aae
I20260809 08:43:27.446499 15341 ts_tablet_manager.cc:1939] T e18a5beab9b54431a46eb4fd84b02aae P 8eceb9371b9843f895deb19fc5c3e6d1: Deleting consensus metadata
I20260809 08:43:27.451330 14589 catalog_manager.cc:4841] TS 1f5b364af6c449598baae1d533432df8 (127.31.63.66:37033): tablet e18a5beab9b54431a46eb4fd84b02aae (table test-table [id=ed007c64596a408a9b15227819a2013c]) successfully deleted
I20260809 08:43:27.451462 14589 catalog_manager.cc:4841] TS 8eceb9371b9843f895deb19fc5c3e6d1 (127.31.63.67:33461): tablet e18a5beab9b54431a46eb4fd84b02aae (table test-table [id=ed007c64596a408a9b15227819a2013c]) successfully deleted
W20260809 08:43:27.456212 14588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:36403 (error 108)
I20260809 08:43:27.456867 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:43:27.456708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:36403 (local address 127.0.0.1:55442)
0809 08:43:27.456780 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 08:43:27.456829 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:36403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 08:43:27.456950 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 14901
W20260809 08:43:27.461035 14588 catalog_manager.cc:4584] TS 63e8ee4d69ef4dae82ea5022aa0db37e (127.31.63.65:36403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e18a5beab9b54431a46eb4fd84b02aae: Network error: Client connection negotiation failed: client connection to 127.31.63.65:36403: connect: Connection refused (error 111)
I20260809 08:43:27.465701 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 15100
I20260809 08:43:27.474965 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 14562
2026-08-09T08:43:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2283 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:43:27 dist-test-slave-gkw7 krb5kdc[15349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:43:27 dist-test-slave-gkw7 krb5kdc[15349](info): set up 2 sockets
Aug 09 08:43:27 dist-test-slave-gkw7 krb5kdc[15349](info): commencing operation
krb5kdc: starting...
W20260809 08:43:28.700201 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:43:28 dist-test-slave-gkw7 krb5kdc[15349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:43:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:43:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/rangeradmin_127.31.63.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/rangeradmin_127.31.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/rangerlookup_127.31.63.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/rangerlookup_127.31.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/HTTP_127.31.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/HTTP_127.31.63.84@KRBTEST.COM.keytab.
I20260809 08:43:28.877254 31997 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskslFlM2/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/postgres -l logfile start
2026-08-09 08:43:31.526 UTC [15394] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 08:43:31.526 UTC [15394] LOG: listening on IPv4 address "127.31.63.84", port 53633
2026-08-09 08:43:31.530 UTC [15394] LOG: listening on Unix socket "/tmp/.s.PGSQL.53633"
2026-08-09 08:43:31.568 UTC [15397] LOG: database system was shut down at 2026-08-09 08:43:30 UTC
2026-08-09 08:43:31.573 UTC [15394] LOG: database system is ready to accept connections
I20260809 08:43:32.683291 31997 mini_postgres.cc:96] Postgres bound to 53633
2026-08-09 08:43:32.691 UTC [15412] FATAL: database "slave" does not exist
127.31.63.84:53633 - accepting connections
I20260809 08:43:32.692140 31997 mini_ranger.cc:161] Starting Ranger...
I20260809 08:43:32.708583 31997 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 08:43:33.215219 31997 mini_ranger.cc:87] Created ranger Postgres database
I20260809 08:43:33.215329 31997 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-admin
2026-08-09 08:43:33,552 [I] DB FLAVOR :POSTGRES
2026-08-09 08:43:33,552 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:43:33,552 [I] Checking connection..
2026-08-09 08:43:33,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 08:43:33,799 [I] Checking connection passed.
2026-08-09 08:43:33,799 [I] --------- Verifying version history table ---------
2026-08-09 08:43:33,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:43:34,061 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:43:34,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:43:34,309 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:43:34,310 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 08:43:34,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 08:43:34.591 UTC [15501] WARNING: there is no transaction in progress
2026-08-09 08:43:34,599 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 08:43:34,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:43:34,889 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 08:43:34,890 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:43:34,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 08:43:35,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 08:43:35,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-09 08:43:35,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:43:35,972 [I] Table x_portal_user does not exist in database ranger
2026-08-09 08:43:35,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:43:36,247 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 08:43:36,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:43:36,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:43:36,779 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 08:43:36,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 08:43:37.786 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.802 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.819 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.834 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.851 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.869 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.935 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.942 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.956 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.966 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.976 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:37.991 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:38.001 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:38.008 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:38.015 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:38.023 UTC [15636] WARNING: there is no transaction in progress
2026-08-09 08:43:38,689 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 08:43:38,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 08:43:38,957 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 08:43:38,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:43:39,236 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 08:43:39,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:43:39,531 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 08:43:39,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:43:39,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:43:40,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-09 08:43:40,304 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 08:43:40,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 08:43:40,581 [I] DB_PATCHES have already been applied
I20260809 08:43:40.588086 31997 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-admin:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskslFlM2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 08:43:40.588178 31997 mini_ranger.cc:193] Using host: 127.31.63.84
I20260809 08:43:40.591961 31997 mini_ranger.cc:227] Ranger admin URL: http://127.31.63.84:56801
Aug 09, 2026 8:43:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:43:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:43:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 8:43:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 8:43:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 8:43:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 8:43:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 08:43:42.089355 31997 mini_ranger.cc:160] Time spent starting Ranger: real 9.397s user 0.000s sys 0.012s
Aug 09, 2026 8:44:02 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 8:44:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 8:44:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 8:44:02 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 8:44:03 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 08:44:04.004137 31997 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/rangerkms_127.31.63.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/rangerkms_127.31.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.63.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.63.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.63.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/HTTP_127.31.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.63.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/HTTP_127.31.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 08:44:04 dist-test-slave-gkw7 krb5kdc[15349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265044, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 08:44:04.094127 31997 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 08:44:04.105626 31997 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 08:44:04.984792 31997 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 08:44:04.984892 31997 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-kms
I20260809 08:44:04.984912 31997 mini_ranger_kms.cc:227] Using postgres at 127.31.63.84:53633
2026-08-09 08:44:05,245 [I] DB FLAVOR :POSTGRES
2026-08-09 08:44:05,245 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:44:05,245 [I] Checking connection
2026-08-09 08:44:05,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 08:44:05,520 [I] connection success
2026-08-09 08:44:05,520 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 08:44:05,520 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 08:44:05,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 08:44:05,772 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 08:44:05,772 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:44:05,772 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 08:44:05,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:53633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 08:44:06,043 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 08:44:06.529228 31997 mini_ranger_kms.cc:310] Created kms service
I20260809 08:44:06.712487 31997 mini_ranger_kms.cc:326] Created kudu user
I20260809 08:44:06.759140 31997 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 08:44:06 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265044, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:44:07.166572 31997 mini_ranger_kms.cc:384] Added ranger policy
I20260809 08:44:07.166709 31997 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-kms:/tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 08:44:07.166731 31997 mini_ranger_kms.cc:240] Using host: 127.31.63.84
I20260809 08:44:07.172111 31997 mini_ranger_kms.cc:276] Ranger KMS PID: 15891
I20260809 08:44:07.172252 31997 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.63.84:42751
Aug 09, 2026 8:44:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:44:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:44:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 8:44:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 8:44:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 8:44:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 08:44:09 dist-test-slave-gkw7 krb5kdc[15349](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265049, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.63.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:44:09.929695 31997 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.836s user 0.024s sys 0.222s
I20260809 08:44:09.929796 31997 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 08:44:09.929878 31997 mini_ranger_kms.cc:401] 127.31.63.84:42751/kms/v1/keys
Aug 09 08:44:10 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265049, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.63.84@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
Aug 09 08:44:10 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265049, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.63.84@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
Aug 09 08:44:13 dist-test-slave-gkw7 krb5kdc[15349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:44:13.056635 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:35517
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:35517
--ranger_config_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:42751/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:44:13.171804 15968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:44:13.172078 15968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:44:13.172120 15968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:44:13.173269 15968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:44:13.173334 15968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:44:13.173358 15968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:44:13.173380 15968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:44:13.173408 15968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:44:13.175143 15968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.63.84:42751/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:35517
--ranger_config_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:35517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:44:13.175426 15968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:44:13.175688 15968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:44:13.180855 15980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:44:13.180939 15975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:44:13.180855 15976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:44:13.181322 15968 server_base.cc:1034] running on GCE node
I20260809 08:44:13.181623 15968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:44:13.181972 15968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:44:13.183130 15968 hybrid_clock.cc:648] HybridClock initialized: now 1786265053183110 us; error 39 us; skew 500 ppm
Aug 09 08:44:13 dist-test-slave-gkw7 krb5kdc[15349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265053, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:44:13.189638 15968 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:44:13.190222 15968 webserver.cc:460] Webserver started at http://127.31.63.126:35115/ using document root <none> and password file <none>
I20260809 08:44:13.190492 15968 fs_manager.cc:359] Metadata directory not provided
I20260809 08:44:13.190533 15968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:44:13.190662 15968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:44:13 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265053, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:44:13.241099 15968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "c9d35b81275e41ecb4b44bd6ce915cf7"
format_stamp: "Formatted at 2026-08-09 08:44:13 on dist-test-slave-gkw7"
server_key: "bb512040dd6d0e69e4e272de2b9f18f6"
server_key_iv: "0fda23513b4d326ff5b8785f72dfe0ac"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:13.241664 15968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "c9d35b81275e41ecb4b44bd6ce915cf7"
format_stamp: "Formatted at 2026-08-09 08:44:13 on dist-test-slave-gkw7"
server_key: "bb512040dd6d0e69e4e272de2b9f18f6"
server_key_iv: "0fda23513b4d326ff5b8785f72dfe0ac"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:13.243294 15968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:44:13.255936 15989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:44:13.256191 15968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:44:13.256309 15968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "c9d35b81275e41ecb4b44bd6ce915cf7"
format_stamp: "Formatted at 2026-08-09 08:44:13 on dist-test-slave-gkw7"
server_key: "bb512040dd6d0e69e4e272de2b9f18f6"
server_key_iv: "0fda23513b4d326ff5b8785f72dfe0ac"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:13.256443 15968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:44:13.283147 15968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:44:13.285391 15968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:44:13.285580 15968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:44:13.290597 15968 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:35517
I20260809 08:44:13.290652 16057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:35517 every 8 connection(s)
I20260809 08:44:13.291044 15968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:44:13.291812 16058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:44:13.294313 16058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7: Bootstrap starting.
I20260809 08:44:13.294716 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 15968
I20260809 08:44:13.294821 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:44:13.295009 16058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7: Neither blocks nor log segments found. Creating new log.
I20260809 08:44:13.295015 31997 external_mini_cluster.cc:1428] Setting key 917b0a6af7472443cec858f401b532dc
I20260809 08:44:13.295415 16058 log.cc:824] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:44:13.296315 16058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7: No bootstrap required, opened a new log
I20260809 08:44:13.298345 16058 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d35b81275e41ecb4b44bd6ce915cf7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35517 } }
I20260809 08:44:13.298519 16058 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:44:13.298550 16058 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9d35b81275e41ecb4b44bd6ce915cf7, State: Initialized, Role: FOLLOWER
I20260809 08:44:13.298683 16058 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d35b81275e41ecb4b44bd6ce915cf7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35517 } }
I20260809 08:44:13.298754 16058 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:44:13.298787 16058 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:44:13.298825 16058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:44:13.299642 16058 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d35b81275e41ecb4b44bd6ce915cf7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35517 } }
I20260809 08:44:13.299754 16058 leader_election.cc:302] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9d35b81275e41ecb4b44bd6ce915cf7; no voters:
I20260809 08:44:13.299986 16058 leader_election.cc:288] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:44:13.300033 16064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:44:13.300297 16064 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [term 1 LEADER]: Becoming Leader. State: Replica: c9d35b81275e41ecb4b44bd6ce915cf7, State: Running, Role: LEADER
I20260809 08:44:13.300384 16058 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:44:13.300441 16064 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d35b81275e41ecb4b44bd6ce915cf7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35517 } }
I20260809 08:44:13.300861 16064 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9d35b81275e41ecb4b44bd6ce915cf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d35b81275e41ecb4b44bd6ce915cf7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35517 } } }
I20260809 08:44:13.301019 16064 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [sys.catalog]: This master's current role is: LEADER
I20260809 08:44:13.301240 16064 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9d35b81275e41ecb4b44bd6ce915cf7. Latest consensus state: current_term: 1 leader_uuid: "c9d35b81275e41ecb4b44bd6ce915cf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d35b81275e41ecb4b44bd6ce915cf7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35517 } } }
I20260809 08:44:13.301385 16064 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7 [sys.catalog]: This master's current role is: LEADER
I20260809 08:44:13.301616 16075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:44:13.302198 16058 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 08:44:13.302217 16075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:44:13 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:44:13.310694 16075 catalog_manager.cc:1347] Generated new cluster ID: add774f0a04d4038815000b5c7f803e3
I20260809 08:44:13.310913 16075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:44:13.312503 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:13.295596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.31.63.126:35517)
0809 08:44:13.295786 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:13.295798 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:13.295977 (+ 179us) server_negotiation.cc:407] Connection header received
0809 08:44:13.296627 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:13.296635 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:13.296690 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:13.296848 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:44:13.297510 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:13.299023 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:13.299856 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:13.300490 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:13.300579 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:13.303871 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:44:13.303875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:44:13.303876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:44:13.303890 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:44:13.309203 (+ 5313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:44:13.309843 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:44:13.309847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:44:13.309848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:44:13.310061 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:44:13.310322 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:44:13.310323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:44:13.310323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:44:13.310989 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:44:13.311294 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:13.311844 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 08:44:13.312200 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260809 08:44:13.325462 16075 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:44:13.326393 16075 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:44:13.331866 16075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7: Generated new TSK 0
I20260809 08:44:13.332171 16075 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:44:14.108115 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:42751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35517
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:44:14.243217 16112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:44:14.243618 16112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:44:14.243707 16112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:44:14.245810 16112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:44:14.245935 16112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:44:14.246031 16112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:44:14.248517 16112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.63.84:42751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:44:14.249037 16112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:44:14.249430 16112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:44:14.258764 16124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:44:14.259029 16128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:44:14.258821 16122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:44:14.259318 16112 server_base.cc:1034] running on GCE node
I20260809 08:44:14.259635 16112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:44:14.259990 16112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:44:14.261305 16112 hybrid_clock.cc:648] HybridClock initialized: now 1786265054261268 us; error 49 us; skew 500 ppm
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:44:14.265172 16112 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:44:14.265780 16112 webserver.cc:460] Webserver started at http://127.31.63.65:38455/ using document root <none> and password file <none>
I20260809 08:44:14.266038 16112 fs_manager.cc:359] Metadata directory not provided
I20260809 08:44:14.266108 16112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:44:14.266301 16112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:44:14.281900 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "e120a088a0494aa4ba4d4cc890a066ea"
format_stamp: "Formatted at 2026-08-09 08:44:14 on dist-test-slave-gkw7"
server_key: "d5c4d713415d06f987036ad170e5eaf5"
server_key_iv: "37148913d72258791468695d690762d0"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:14.282531 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "e120a088a0494aa4ba4d4cc890a066ea"
format_stamp: "Formatted at 2026-08-09 08:44:14 on dist-test-slave-gkw7"
server_key: "d5c4d713415d06f987036ad170e5eaf5"
server_key_iv: "37148913d72258791468695d690762d0"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:14.288630 16112 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260809 08:44:14.309672 16149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:44:14.310683 16112 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260809 08:44:14.310810 16112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "e120a088a0494aa4ba4d4cc890a066ea"
format_stamp: "Formatted at 2026-08-09 08:44:14 on dist-test-slave-gkw7"
server_key: "d5c4d713415d06f987036ad170e5eaf5"
server_key_iv: "37148913d72258791468695d690762d0"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:14.310943 16112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:44:14.335076 16112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:44:14.337679 16112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:44:14.337898 16112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:44:14.338213 16112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:44:14.338671 16112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:44:14.338768 16112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:44:14.338873 16112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:44:14.338950 16112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:44:14.348143 16112 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:35545
I20260809 08:44:14.348713 16112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:44:14.349227 16306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:35545 every 8 connection(s)
I20260809 08:44:14.356408 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 16112
I20260809 08:44:14.356527 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:44:14.356745 31997 external_mini_cluster.cc:1428] Setting key ffeefd396b772cd3ad2940fb5acfc0df
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:44:14.374629 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.350090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:35517 (local address 127.31.63.65:55023)
0809 08:44:14.353641 (+ 3551us) negotiation.cc:107] Waiting for socket to connect
0809 08:44:14.353699 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 08:44:14.354421 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:44:14.360801 (+ 6380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:44:14.360807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:44:14.361117 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:44:14.361428 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.361435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.362653 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.362655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.364151 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.364155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.364739 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.364740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.364895 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.365666 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:44:14.365678 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:44:14.367641 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:44:14.369978 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:44:14.369986 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:44:14.369988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:44:14.370324 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:44:14.372897 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:44:14.372899 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:44:14.372899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:44:14.373075 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:44:14.373842 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:44:14.373850 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:44:14.374032 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 08:44:14.374130 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 08:44:14.374316 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3495,"thread_start_us":107,"threads_started":1}
I20260809 08:44:14.375144 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.350112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:55023 (local address 127.31.63.126:35517)
0809 08:44:14.360497 (+ 10385us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:14.360499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:14.360510 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:44:14.360537 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:14.360538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:14.360553 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:14.360630 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:44:14.361606 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.362526 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.364268 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.364577 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.364656 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.367770 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:44:14.367772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:44:14.367773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:44:14.367782 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:44:14.369875 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:44:14.372745 (+ 2870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:44:14.372746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:44:14.372747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:44:14.372823 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:44:14.373574 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:44:14.373575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:44:14.373575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:44:14.373745 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:44:14.374899 (+ 1154us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:14.374948 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 08:44:14.375046 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10356,"thread_start_us":77,"threads_started":1}
I20260809 08:44:14.375795 16308 heartbeater.cc:347] Connected to a master server at 127.31.63.126:35517
I20260809 08:44:14.375896 16308 heartbeater.cc:464] Registering TS with master...
I20260809 08:44:14.376096 16308 heartbeater.cc:511] Master 127.31.63.126:35517 requested a full tablet report, sending...
I20260809 08:44:14.376637 16013 ts_manager.cc:194] Registered new tserver with Master: e120a088a0494aa4ba4d4cc890a066ea (127.31.63.65:35545)
I20260809 08:44:14.377558 16013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:55023
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:44:14.411156 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:42751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35517
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 08:44:14.553643 16320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:44:14.554028 16320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:44:14.554119 16320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:44:14.555953 16320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:44:14.556087 16320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:44:14.556174 16320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:44:14.558635 16320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.63.84:42751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:44:14.559103 16320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:44:14.559451 16320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:44:14.566243 16330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:44:14.566241 16331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:44:14.566279 16336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:44:14.567421 16320 server_base.cc:1034] running on GCE node
I20260809 08:44:14.567637 16320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:44:14.567895 16320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:44:14.569044 16320 hybrid_clock.cc:648] HybridClock initialized: now 1786265054569026 us; error 36 us; skew 500 ppm
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:44:14.571934 16320 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:44:14.572499 16320 webserver.cc:460] Webserver started at http://127.31.63.66:33187/ using document root <none> and password file <none>
I20260809 08:44:14.572824 16320 fs_manager.cc:359] Metadata directory not provided
I20260809 08:44:14.572899 16320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:44:14.573053 16320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:44:14.587112 16320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "b182777c02d94f9587b4614bcd4dab96"
format_stamp: "Formatted at 2026-08-09 08:44:14 on dist-test-slave-gkw7"
server_key: "968dc8701b5e6f624083cec7453447c0"
server_key_iv: "c93dd459f43421238bcbdcbbb28e78d0"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:14.587575 16320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "b182777c02d94f9587b4614bcd4dab96"
format_stamp: "Formatted at 2026-08-09 08:44:14 on dist-test-slave-gkw7"
server_key: "968dc8701b5e6f624083cec7453447c0"
server_key_iv: "c93dd459f43421238bcbdcbbb28e78d0"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:14.589080 16320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:44:14.609263 16344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:44:14.609556 16320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 08:44:14.609714 16320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "b182777c02d94f9587b4614bcd4dab96"
format_stamp: "Formatted at 2026-08-09 08:44:14 on dist-test-slave-gkw7"
server_key: "968dc8701b5e6f624083cec7453447c0"
server_key_iv: "c93dd459f43421238bcbdcbbb28e78d0"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:14.609910 16320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:44:14.643380 16320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:44:14.645601 16320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:44:14.645768 16320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:44:14.646020 16320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:44:14.646437 16320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:44:14.646492 16320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:44:14.646557 16320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:44:14.646588 16320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:44:14.656373 16320 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:33385
I20260809 08:44:14.656868 16320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:44:14.657423 16510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:33385 every 8 connection(s)
I20260809 08:44:14.659897 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 16320
I20260809 08:44:14.660061 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:44:14.660306 31997 external_mini_cluster.cc:1428] Setting key bca7e25a317445486aa9e4ed6f1e6dea
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:44:14.670513 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.657855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:58155 (local address 127.31.63.126:35517)
0809 08:44:14.657914 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:14.657917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:14.658193 (+ 276us) server_negotiation.cc:407] Connection header received
0809 08:44:14.658825 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:14.658827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:14.658848 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:14.658928 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:44:14.659663 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.660577 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.662303 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.662694 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.662750 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.666028 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:44:14.666030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:44:14.666031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:44:14.666040 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:44:14.668320 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:44:14.668995 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:44:14.668997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:44:14.668998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:44:14.669076 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:44:14.669464 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:44:14.669465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:44:14.669465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:44:14.669694 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:44:14.669760 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:14.670247 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 08:44:14.670364 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:44:14.671136 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.657832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:35517 (local address 127.31.63.66:58155)
0809 08:44:14.658044 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 08:44:14.658090 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:44:14.658700 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:44:14.659013 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:44:14.659017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:44:14.659267 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:44:14.659531 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.659536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.660714 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.660715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.662173 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.662177 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.662869 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.662870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.663036 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.663796 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:44:14.663810 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:44:14.665894 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:44:14.668452 (+ 2558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:44:14.668462 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:44:14.668465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:44:14.668872 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:44:14.669181 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:44:14.669182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:44:14.669182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:44:14.669357 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:44:14.669831 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:44:14.669841 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:44:14.670040 (+ 199us) client_negotiation.cc:769] Sending connection context
0809 08:44:14.670653 (+ 613us) client_negotiation.cc:240] Negotiation successful
0809 08:44:14.670859 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":124,"threads_started":1}
I20260809 08:44:14.671711 16511 heartbeater.cc:347] Connected to a master server at 127.31.63.126:35517
I20260809 08:44:14.671828 16511 heartbeater.cc:464] Registering TS with master...
I20260809 08:44:14.672070 16511 heartbeater.cc:511] Master 127.31.63.126:35517 requested a full tablet report, sending...
I20260809 08:44:14.672521 16013 ts_manager.cc:194] Registered new tserver with Master: b182777c02d94f9587b4614bcd4dab96 (127.31.63.66:33385)
I20260809 08:44:14.672959 16013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:58155
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:44:14.694743 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:42751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35517
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:44:14.808027 16522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:44:14.808418 16522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:44:14.808523 16522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:44:14.810323 16522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:44:14.810432 16522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:44:14.810532 16522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:44:14.812714 16522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.63.84:42751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:35517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:44:14.813192 16522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:44:14.813512 16522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:44:14.820478 16529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:44:14.820482 16528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:44:14.821283 16531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:44:14.821897 16522 server_base.cc:1034] running on GCE node
I20260809 08:44:14.822134 16522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:44:14.822398 16522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:44:14.823573 16522 hybrid_clock.cc:648] HybridClock initialized: now 1786265054823540 us; error 47 us; skew 500 ppm
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:44:14.827293 16522 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:44:14.827885 16522 webserver.cc:460] Webserver started at http://127.31.63.67:38711/ using document root <none> and password file <none>
I20260809 08:44:14.828161 16522 fs_manager.cc:359] Metadata directory not provided
I20260809 08:44:14.828222 16522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:44:14.828356 16522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:44:14.849712 16522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "829e434278ad4cb597d6e4194a0a0a01"
format_stamp: "Formatted at 2026-08-09 08:44:14 on dist-test-slave-gkw7"
server_key: "70ad7895e707fa740bd9237e06473711"
server_key_iv: "2df6970eaab53d8ecb380d6583c33874"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:14.850348 16522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "829e434278ad4cb597d6e4194a0a0a01"
format_stamp: "Formatted at 2026-08-09 08:44:14 on dist-test-slave-gkw7"
server_key: "70ad7895e707fa740bd9237e06473711"
server_key_iv: "2df6970eaab53d8ecb380d6583c33874"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:14.853986 16522 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 08:44:14.880905 16538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:44:14.881115 16522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:44:14.881219 16522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "829e434278ad4cb597d6e4194a0a0a01"
format_stamp: "Formatted at 2026-08-09 08:44:14 on dist-test-slave-gkw7"
server_key: "70ad7895e707fa740bd9237e06473711"
server_key_iv: "2df6970eaab53d8ecb380d6583c33874"
server_key_version: "kuduclusterkey@0"
I20260809 08:44:14.881327 16522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:44:14.910480 16522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:44:14.912739 16522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:44:14.912904 16522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:44:14.913167 16522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:44:14.913596 16522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:44:14.913658 16522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:44:14.913697 16522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:44:14.913722 16522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:44:14.922847 16522 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:40153
I20260809 08:44:14.923292 16522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:44:14.923892 16694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:40153 every 8 connection(s)
I20260809 08:44:14.925209 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 16522
I20260809 08:44:14.925326 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:44:14.925510 31997 external_mini_cluster.cc:1428] Setting key 5a8752bfcd2dd05e21f309542c6d1d3b
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:44:14.935271 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.924104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:32909 (local address 127.31.63.126:35517)
0809 08:44:14.924152 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:14.924154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:14.924585 (+ 431us) server_negotiation.cc:407] Connection header received
0809 08:44:14.926027 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:14.926028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:14.926041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:14.926085 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:44:14.926972 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.927619 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.929159 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.929474 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.929519 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.932171 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:44:14.932173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:44:14.932173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:44:14.932181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:44:14.933863 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:44:14.934270 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:44:14.934271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:44:14.934272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:44:14.934330 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:44:14.934616 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:44:14.934617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:44:14.934618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:44:14.934739 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:44:14.934815 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:14.935096 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 08:44:14.935178 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:44:14.935467 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.924132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:35517 (local address 127.31.63.67:32909)
0809 08:44:14.924446 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 08:44:14.924494 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:44:14.925854 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:44:14.926467 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:44:14.926471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:44:14.926678 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:44:14.926879 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.926884 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.927735 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.927736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.929052 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.929055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.929653 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.929654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.929796 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.930443 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:44:14.930454 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:44:14.932075 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:44:14.933947 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:44:14.933954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:44:14.933957 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:44:14.934179 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:44:14.934406 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:44:14.934407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:44:14.934407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:44:14.934554 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:44:14.934806 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:44:14.934813 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:44:14.934957 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 08:44:14.935065 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:44:14.935214 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":84,"threads_started":1}
I20260809 08:44:14.936075 16695 heartbeater.cc:347] Connected to a master server at 127.31.63.126:35517
I20260809 08:44:14.936189 16695 heartbeater.cc:464] Registering TS with master...
I20260809 08:44:14.936372 16695 heartbeater.cc:511] Master 127.31.63.126:35517 requested a full tablet report, sending...
I20260809 08:44:14.936764 16013 ts_manager.cc:194] Registered new tserver with Master: 829e434278ad4cb597d6e4194a0a0a01 (127.31.63.67:40153)
I20260809 08:44:14.937151 16013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:32909
I20260809 08:44:14.937620 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:44:14.949802 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.939393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55706 (local address 127.31.63.126:35517)
0809 08:44:14.939434 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:14.939436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:14.939605 (+ 169us) server_negotiation.cc:407] Connection header received
0809 08:44:14.939679 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:14.939680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:14.939691 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:14.939739 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:44:14.940293 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.940928 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.942227 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.942584 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.942635 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.946774 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:44:14.946775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:44:14.946776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:44:14.946783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:44:14.948488 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:44:14.948900 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:44:14.948901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:44:14.948902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:44:14.948958 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:44:14.949148 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:44:14.949149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:44:14.949149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:44:14.949275 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:44:14.949347 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:14.949604 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 08:44:14.949685 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:44:14.952322 16013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:44:14.955391 16013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 09 08:44:14 dist-test-slave-gkw7 krb5kdc[15349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265054, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:44:14.982681 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.977016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:33385 (local address 127.0.0.1:52386)
0809 08:44:14.978052 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0809 08:44:14.978062 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:44:14.978108 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:44:14.978499 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:44:14.978500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:44:14.978502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:44:14.978536 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.978537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.979803 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.979805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.981180 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.981182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.982342 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.982343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.982423 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.982437 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:44:14.982504 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:44:14.982577 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"mutex_wait_us":5,"thread_start_us":95,"threads_started":1}
I20260809 08:44:14.982908 16708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.977016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40153 (local address 127.0.0.1:37058)
0809 08:44:14.977782 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0809 08:44:14.977831 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 08:44:14.977897 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:44:14.978372 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:44:14.978376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:44:14.978378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:44:14.978463 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.978465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.980241 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.980242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.981300 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.981302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.982674 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.982675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.982735 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.982738 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:44:14.982791 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:44:14.982834 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":101,"threads_started":1}
I20260809 08:44:14.983037 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.977321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52386 (local address 127.31.63.66:33385)
0809 08:44:14.977663 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:14.977668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:14.978169 (+ 501us) server_negotiation.cc:407] Connection header received
0809 08:44:14.978213 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:14.978216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:14.978232 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:14.978299 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:44:14.978625 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.979714 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.981301 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.982147 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.982192 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.982281 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:14.982891 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 08:44:14.982936 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":123,"threads_started":1}
I20260809 08:44:14.983209 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.977520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37058 (local address 127.31.63.67:40153)
0809 08:44:14.978116 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:14.978121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:14.978132 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:44:14.978176 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:14.978179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:14.978195 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:14.978271 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:44:14.978546 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.980151 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.981402 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.982512 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.982569 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.982642 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:14.983085 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 08:44:14.983134 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":116,"threads_started":1}
I20260809 08:44:14.983722 16413 tablet_service.cc:1467] Processing CreateTablet for tablet 0a713cb9ee3642d29f44aff8e39875d4 (DEFAULT_TABLE table=test-table [id=9e0a3ec0d9844610b56ee0a0addd5028]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:44:14.983722 16607 tablet_service.cc:1467] Processing CreateTablet for tablet 0a713cb9ee3642d29f44aff8e39875d4 (DEFAULT_TABLE table=test-table [id=9e0a3ec0d9844610b56ee0a0addd5028]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:44:14.984129 16607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a713cb9ee3642d29f44aff8e39875d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:44:14.984129 16413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a713cb9ee3642d29f44aff8e39875d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:44:14.988430 16716 tablet_bootstrap.cc:492] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: Bootstrap starting.
I20260809 08:44:14.989614 16716 tablet_bootstrap.cc:654] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: Neither blocks nor log segments found. Creating new log.
I20260809 08:44:14.990314 16716 log.cc:824] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: Log is configured to *not* fsync() on all Append() calls
I20260809 08:44:14.990715 16717 tablet_bootstrap.cc:492] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: Bootstrap starting.
I20260809 08:44:14.991744 16717 tablet_bootstrap.cc:654] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: Neither blocks nor log segments found. Creating new log.
I20260809 08:44:14.992426 16717 log.cc:824] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: Log is configured to *not* fsync() on all Append() calls
I20260809 08:44:14.992692 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.977406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57514 (local address 127.31.63.65:35545)
0809 08:44:14.987252 (+ 9846us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:14.987257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:14.987273 (+ 16us) server_negotiation.cc:407] Connection header received
0809 08:44:14.987318 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:14.987321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:14.987342 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:14.987457 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:44:14.987716 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.988904 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.990337 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.991588 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.992159 (+ 571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.992235 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:14.992564 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 08:44:14.992621 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9797,"thread_start_us":79,"threads_started":1}
I20260809 08:44:14.992996 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:14.977277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35545 (local address 127.0.0.1:57514)
0809 08:44:14.978392 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0809 08:44:14.978402 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:44:14.978444 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:44:14.987554 (+ 9110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:44:14.987556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:44:14.987559 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:44:14.987605 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.987607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.989022 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.989023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.990238 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:14.990241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:14.992274 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:14.992275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:14.992343 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:14.992348 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:44:14.992402 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 08:44:14.992446 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1100,"thread_start_us":73,"threads_started":1}
I20260809 08:44:14.993785 16716 tablet_bootstrap.cc:492] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: No bootstrap required, opened a new log
I20260809 08:44:14.993912 16716 ts_tablet_manager.cc:1397] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260809 08:44:14.993947 16219 tablet_service.cc:1467] Processing CreateTablet for tablet 0a713cb9ee3642d29f44aff8e39875d4 (DEFAULT_TABLE table=test-table [id=9e0a3ec0d9844610b56ee0a0addd5028]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:44:14.994333 16219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a713cb9ee3642d29f44aff8e39875d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:44:14.996688 16716 raft_consensus.cc:348] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 } } peers { permanent_uuid: "e120a088a0494aa4ba4d4cc890a066ea" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35545 } } peers { permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 } }
I20260809 08:44:14.996961 16716 raft_consensus.cc:374] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:44:14.997057 16716 raft_consensus.cc:729] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b182777c02d94f9587b4614bcd4dab96, State: Initialized, Role: FOLLOWER
I20260809 08:44:14.997287 16716 consensus_queue.cc:260] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 } } peers { permanent_uuid: "e120a088a0494aa4ba4d4cc890a066ea" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35545 } } peers { permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 } }
I20260809 08:44:14.997711 16716 ts_tablet_manager.cc:1428] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 08:44:14.997928 16511 heartbeater.cc:503] Master 127.31.63.126:35517 was elected leader, sending a full tablet report...
I20260809 08:44:14.999209 16722 tablet_bootstrap.cc:492] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea: Bootstrap starting.
I20260809 08:44:15.000254 16722 tablet_bootstrap.cc:654] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea: Neither blocks nor log segments found. Creating new log.
I20260809 08:44:15.000722 16722 log.cc:824] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea: Log is configured to *not* fsync() on all Append() calls
I20260809 08:44:15.001878 16722 tablet_bootstrap.cc:492] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea: No bootstrap required, opened a new log
I20260809 08:44:15.002025 16722 ts_tablet_manager.cc:1397] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:44:15.004853 16722 raft_consensus.cc:348] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 } } peers { permanent_uuid: "e120a088a0494aa4ba4d4cc890a066ea" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35545 } } peers { permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 } }
I20260809 08:44:15.005095 16722 raft_consensus.cc:374] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:44:15.005185 16722 raft_consensus.cc:729] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e120a088a0494aa4ba4d4cc890a066ea, State: Initialized, Role: FOLLOWER
I20260809 08:44:15.005369 16722 consensus_queue.cc:260] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 } } peers { permanent_uuid: "e120a088a0494aa4ba4d4cc890a066ea" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35545 } } peers { permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 } }
I20260809 08:44:15.009506 16717 tablet_bootstrap.cc:492] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: No bootstrap required, opened a new log
I20260809 08:44:15.009617 16717 ts_tablet_manager.cc:1397] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.001s
I20260809 08:44:15.011678 16717 raft_consensus.cc:348] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 } } peers { permanent_uuid: "e120a088a0494aa4ba4d4cc890a066ea" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35545 } } peers { permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 } }
I20260809 08:44:15.011852 16717 raft_consensus.cc:374] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:44:15.011889 16717 raft_consensus.cc:729] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829e434278ad4cb597d6e4194a0a0a01, State: Initialized, Role: FOLLOWER
I20260809 08:44:15.012185 16722 ts_tablet_manager.cc:1428] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260809 08:44:15.012395 16308 heartbeater.cc:503] Master 127.31.63.126:35517 was elected leader, sending a full tablet report...
I20260809 08:44:15.012925 16717 consensus_queue.cc:260] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 } } peers { permanent_uuid: "e120a088a0494aa4ba4d4cc890a066ea" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35545 } } peers { permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 } }
I20260809 08:44:15.013252 16717 ts_tablet_manager.cc:1428] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 08:44:15.013379 16695 heartbeater.cc:503] Master 127.31.63.126:35517 was elected leader, sending a full tablet report...
I20260809 08:44:15.098965 16724 raft_consensus.cc:482] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:44:15.099155 16724 raft_consensus.cc:504] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 } } peers { permanent_uuid: "e120a088a0494aa4ba4d4cc890a066ea" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35545 } } peers { permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 } }
I20260809 08:44:15.100095 16724 leader_election.cc:288] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 829e434278ad4cb597d6e4194a0a0a01 (127.31.63.67:40153), b182777c02d94f9587b4614bcd4dab96 (127.31.63.66:33385)
W20260809 08:44:15.100968 16309 tablet.cc:2369] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:44:15.104104 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:15.100016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:36487 (local address 127.31.63.66:33385)
0809 08:44:15.100068 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:15.100070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:15.100476 (+ 406us) server_negotiation.cc:407] Connection header received
0809 08:44:15.100581 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:15.100583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:15.100595 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:15.100625 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:44:15.100852 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:15.101589 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:15.103128 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:15.103693 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:15.103741 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:15.103752 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:15.103982 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 08:44:15.104033 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:44:15.104171 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:15.100038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:59935 (local address 127.31.63.67:40153)
0809 08:44:15.100085 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:15.100087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:15.100328 (+ 241us) server_negotiation.cc:407] Connection header received
0809 08:44:15.100387 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:15.100388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:15.100399 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:15.100453 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:44:15.100714 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:15.101619 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:15.103215 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:15.103736 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:15.103776 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:15.103783 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:15.104044 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 08:44:15.104084 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:44:15.104383 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:15.099757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:33385 (local address 127.31.63.65:36487)
0809 08:44:15.100423 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 08:44:15.100432 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:44:15.100501 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:44:15.100705 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:44:15.100705 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:44:15.100707 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:44:15.100773 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:15.100775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:15.101743 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:15.101744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:15.103022 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:15.103025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:15.103817 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:15.103818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:15.103894 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:15.103899 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:44:15.104222 (+ 323us) client_negotiation.cc:240] Negotiation successful
0809 08:44:15.104270 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":21,"thread_start_us":104,"threads_started":1}
I20260809 08:44:15.104720 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:15.099757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40153 (local address 127.31.63.65:59935)
0809 08:44:15.100248 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 08:44:15.100259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:44:15.100327 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:44:15.100522 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:44:15.100523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:44:15.100525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:44:15.100624 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:15.100626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:15.101968 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:15.101969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:15.103107 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:44:15.103110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:44:15.103874 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:44:15.103874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:44:15.103963 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:15.103966 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:44:15.104582 (+ 616us) client_negotiation.cc:240] Negotiation successful
0809 08:44:15.104631 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":123,"threads_started":1}
I20260809 08:44:15.104758 16439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a713cb9ee3642d29f44aff8e39875d4" candidate_uuid: "e120a088a0494aa4ba4d4cc890a066ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b182777c02d94f9587b4614bcd4dab96" is_pre_election: true
I20260809 08:44:15.104933 16439 raft_consensus.cc:2454] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e120a088a0494aa4ba4d4cc890a066ea in term 0.
I20260809 08:44:15.105019 16633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a713cb9ee3642d29f44aff8e39875d4" candidate_uuid: "e120a088a0494aa4ba4d4cc890a066ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829e434278ad4cb597d6e4194a0a0a01" is_pre_election: true
I20260809 08:44:15.105173 16633 raft_consensus.cc:2454] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e120a088a0494aa4ba4d4cc890a066ea in term 0.
I20260809 08:44:15.105175 16150 leader_election.cc:302] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b182777c02d94f9587b4614bcd4dab96, e120a088a0494aa4ba4d4cc890a066ea; no voters:
I20260809 08:44:15.105297 16724 raft_consensus.cc:2781] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:44:15.105347 16724 raft_consensus.cc:482] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:44:15.105374 16724 raft_consensus.cc:3037] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:44:15.106596 16724 raft_consensus.cc:504] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 } } peers { permanent_uuid: "e120a088a0494aa4ba4d4cc890a066ea" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35545 } } peers { permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 } }
I20260809 08:44:15.106774 16724 leader_election.cc:288] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [CANDIDATE]: Term 1 election: Requested vote from peers 829e434278ad4cb597d6e4194a0a0a01 (127.31.63.67:40153), b182777c02d94f9587b4614bcd4dab96 (127.31.63.66:33385)
I20260809 08:44:15.107018 16633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a713cb9ee3642d29f44aff8e39875d4" candidate_uuid: "e120a088a0494aa4ba4d4cc890a066ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "829e434278ad4cb597d6e4194a0a0a01"
I20260809 08:44:15.107028 16439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a713cb9ee3642d29f44aff8e39875d4" candidate_uuid: "e120a088a0494aa4ba4d4cc890a066ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b182777c02d94f9587b4614bcd4dab96"
I20260809 08:44:15.107098 16439 raft_consensus.cc:3037] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:44:15.107098 16633 raft_consensus.cc:3037] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:44:15.108028 16439 raft_consensus.cc:2454] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e120a088a0494aa4ba4d4cc890a066ea in term 1.
I20260809 08:44:15.108028 16633 raft_consensus.cc:2454] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e120a088a0494aa4ba4d4cc890a066ea in term 1.
I20260809 08:44:15.108235 16155 leader_election.cc:302] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 829e434278ad4cb597d6e4194a0a0a01, e120a088a0494aa4ba4d4cc890a066ea; no voters:
I20260809 08:44:15.108489 16724 raft_consensus.cc:2781] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:44:15.108767 16724 raft_consensus.cc:686] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [term 1 LEADER]: Becoming Leader. State: Replica: e120a088a0494aa4ba4d4cc890a066ea, State: Running, Role: LEADER
I20260809 08:44:15.108867 16724 consensus_queue.cc:237] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 } } peers { permanent_uuid: "e120a088a0494aa4ba4d4cc890a066ea" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35545 } } peers { permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 } }
I20260809 08:44:15.110009 16013 catalog_manager.cc:5495] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea reported cstate change: term changed from 0 to 1, leader changed from <none> to e120a088a0494aa4ba4d4cc890a066ea (127.31.63.65). New cstate: current_term: 1 leader_uuid: "e120a088a0494aa4ba4d4cc890a066ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e120a088a0494aa4ba4d4cc890a066ea" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 } health_report { overall_health: UNKNOWN } } }
I20260809 08:44:15.129930 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:15.124808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.31.63.65:35545)
0809 08:44:15.126515 (+ 1707us) server_negotiation.cc:206] Beginning negotiation
0809 08:44:15.126517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:44:15.126529 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:44:15.126573 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:44:15.126574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:44:15.126598 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:44:15.126648 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:44:15.126888 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:15.127397 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:15.128659 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:44:15.128975 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:44:15.129044 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:44:15.129278 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:44:15.129418 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:44:15.129498 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:44:15.129797 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 08:44:15.129825 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1675}
I20260809 08:44:15.132004 16439 raft_consensus.cc:1261] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96 [term 1 FOLLOWER]: Refusing update from remote peer e120a088a0494aa4ba4d4cc890a066ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:44:15.132400 16724 consensus_queue.cc:1035] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b182777c02d94f9587b4614bcd4dab96" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:44:15.132988 16633 raft_consensus.cc:1261] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01 [term 1 FOLLOWER]: Refusing update from remote peer e120a088a0494aa4ba4d4cc890a066ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:44:15.133270 16724 consensus_queue.cc:1035] T 0a713cb9ee3642d29f44aff8e39875d4 P e120a088a0494aa4ba4d4cc890a066ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "829e434278ad4cb597d6e4194a0a0a01" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:44:15.135488 16734 mvcc.cc:205] Tried to move back new op lower bound from 7316541665817907200 to 7316541665726545920. Current Snapshot: MvccSnapshot[applied={T|T < 7316541665817907200}]
I20260809 08:44:15.137177 16735 mvcc.cc:205] Tried to move back new op lower bound from 7316541665817907200 to 7316541665726545920. Current Snapshot: MvccSnapshot[applied={T|T < 7316541665817907200 or (T in {7316541665817907200})}]
I20260809 08:44:15.137974 16739 mvcc.cc:205] Tried to move back new op lower bound from 7316541665817907200 to 7316541665726545920. Current Snapshot: MvccSnapshot[applied={T|T < 7316541665817907200}]
I20260809 08:44:15.140534 16013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:44:15.140641 16013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:44:15.141497 16013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9d35b81275e41ecb4b44bd6ce915cf7: Sending DeleteTablet for 3 replicas of tablet 0a713cb9ee3642d29f44aff8e39875d4
I20260809 08:44:15.141983 16413 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a713cb9ee3642d29f44aff8e39875d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:44:15 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:52386
I20260809 08:44:15.142144 16607 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a713cb9ee3642d29f44aff8e39875d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:44:15 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:37058
I20260809 08:44:15.142374 16219 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a713cb9ee3642d29f44aff8e39875d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:44:15 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:57514
I20260809 08:44:15.143042 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 16112
I20260809 08:44:15.152820 16750 tablet_replica.cc:330] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: stopping tablet replica
I20260809 08:44:15.157742 16750 raft_consensus.cc:2229] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:44:15.158373 16750 raft_consensus.cc:2258] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:44:15.159003 16750 ts_tablet_manager.cc:1905] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:44:15.160686 16749 tablet_replica.cc:330] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: stopping tablet replica
I20260809 08:44:15.160840 16749 raft_consensus.cc:2229] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:44:15.161002 16749 raft_consensus.cc:2258] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:44:15.161628 16749 ts_tablet_manager.cc:1905] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:44:15.161669 16750 ts_tablet_manager.cc:1918] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:44:15.161898 16750 log.cc:1196] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/0a713cb9ee3642d29f44aff8e39875d4
I20260809 08:44:15.162690 16750 ts_tablet_manager.cc:1939] T 0a713cb9ee3642d29f44aff8e39875d4 P 829e434278ad4cb597d6e4194a0a0a01: Deleting consensus metadata
I20260809 08:44:15.163317 15995 catalog_manager.cc:4841] TS 829e434278ad4cb597d6e4194a0a0a01 (127.31.63.67:40153): tablet 0a713cb9ee3642d29f44aff8e39875d4 (table test-table [id=9e0a3ec0d9844610b56ee0a0addd5028]) successfully deleted
I20260809 08:44:15.163625 16749 ts_tablet_manager.cc:1918] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:44:15.163720 16749 log.cc:1196] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/0a713cb9ee3642d29f44aff8e39875d4
I20260809 08:44:15.164036 16749 ts_tablet_manager.cc:1939] T 0a713cb9ee3642d29f44aff8e39875d4 P b182777c02d94f9587b4614bcd4dab96: Deleting consensus metadata
I20260809 08:44:15.164529 15990 catalog_manager.cc:4841] TS b182777c02d94f9587b4614bcd4dab96 (127.31.63.66:33385): tablet 0a713cb9ee3642d29f44aff8e39875d4 (table test-table [id=9e0a3ec0d9844610b56ee0a0addd5028]) successfully deleted
W20260809 08:44:15.165508 15995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:35545 (error 108)
I20260809 08:44:15.166083 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:44:15.165959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35545 (local address 127.0.0.1:57534)
0809 08:44:15.166001 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 08:44:15.166041 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 08:44:15.166206 15995 catalog_manager.cc:4584] TS e120a088a0494aa4ba4d4cc890a066ea (127.31.63.65:35545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a713cb9ee3642d29f44aff8e39875d4: Network error: Client connection negotiation failed: client connection to 127.31.63.65:35545: connect: Connection refused (error 111)
I20260809 08:44:15.166538 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 16320
I20260809 08:44:15.178736 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 16522
I20260809 08:44:15.189391 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 15968
I20260809 08:44:15.201443 31997 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 08:44:15.552433 31997 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 08:44:15.552551 31997 mini_ranger.cc:66] Stopping Ranger...
I20260809 08:44:15.907770 31997 mini_ranger.cc:68] Stopped Ranger
2026-08-09 08:44:15.907 UTC [15394] LOG: received smart shutdown request
2026-08-09 08:44:15.912 UTC [15394] LOG: background worker "logical replication launcher" (PID 15410) exited with exit code 1
2026-08-09 08:44:15.912 UTC [15398] LOG: shutting down
2026-08-09 08:44:16.079 UTC [15394] LOG: database system is shut down
2026-08-09T08:44:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48721 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:44:16 dist-test-slave-gkw7 krb5kdc[16766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:44:16 dist-test-slave-gkw7 krb5kdc[16766](info): set up 2 sockets
Aug 09 08:44:16 dist-test-slave-gkw7 krb5kdc[16766](info): commencing operation
krb5kdc: starting...
W20260809 08:44:17.450569 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:44:17 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:44:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:44:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/rangeradmin_127.31.63.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/rangeradmin_127.31.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/rangerlookup_127.31.63.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/rangerlookup_127.31.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/HTTP_127.31.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/HTTP_127.31.63.84@KRBTEST.COM.keytab.
I20260809 08:44:17.628280 31997 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskslFlM2/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/postgres -l logfile start
2026-08-09 08:44:20.218 UTC [16806] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 08:44:20.218 UTC [16806] LOG: listening on IPv4 address "127.31.63.84", port 60963
2026-08-09 08:44:20.222 UTC [16806] LOG: listening on Unix socket "/tmp/.s.PGSQL.60963"
2026-08-09 08:44:20.244 UTC [16809] LOG: database system was shut down at 2026-08-09 08:44:18 UTC
2026-08-09 08:44:20.247 UTC [16806] LOG: database system is ready to accept connections
I20260809 08:44:21.419781 31997 mini_postgres.cc:96] Postgres bound to 60963
2026-08-09 08:44:21.427 UTC [16822] FATAL: database "slave" does not exist
127.31.63.84:60963 - accepting connections
I20260809 08:44:21.427680 31997 mini_ranger.cc:161] Starting Ranger...
I20260809 08:44:21.445406 31997 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 08:44:21.941596 31997 mini_ranger.cc:87] Created ranger Postgres database
I20260809 08:44:21.941679 31997 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-admin
2026-08-09 08:44:22,241 [I] DB FLAVOR :POSTGRES
2026-08-09 08:44:22,242 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:44:22,242 [I] Checking connection..
2026-08-09 08:44:22,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 08:44:22,509 [I] Checking connection passed.
2026-08-09 08:44:22,510 [I] --------- Verifying version history table ---------
2026-08-09 08:44:22,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:44:22,788 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:44:22,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:44:23,051 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:44:23,052 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 08:44:23,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 08:44:23.326 UTC [16907] WARNING: there is no transaction in progress
2026-08-09 08:44:23,337 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 08:44:23,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:44:23,634 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 08:44:23,635 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:44:23,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 08:44:23,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 08:44:24,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-09 08:44:24,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:44:24,718 [I] Table x_portal_user does not exist in database ranger
2026-08-09 08:44:24,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:44:24,969 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 08:44:24,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:44:25,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:44:25,528 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 08:44:25,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 08:44:26.533 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.550 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.566 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.582 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.598 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.617 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.677 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.684 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.700 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.710 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.720 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.733 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.743 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.749 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.756 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:26.763 UTC [17064] WARNING: there is no transaction in progress
2026-08-09 08:44:27,428 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 08:44:27,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 08:44:27,683 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 08:44:27,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:44:27,930 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 08:44:27,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:44:28,202 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 08:44:28,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:44:28,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:44:28,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-09 08:44:28,997 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 08:44:28,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 08:44:29,262 [I] DB_PATCHES have already been applied
I20260809 08:44:29.268246 31997 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-admin:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskslFlM2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 08:44:29.268317 31997 mini_ranger.cc:193] Using host: 127.31.63.84
I20260809 08:44:29.272562 31997 mini_ranger.cc:227] Ranger admin URL: http://127.31.63.84:45411
Aug 09, 2026 8:44:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:44:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:44:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 8:44:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 8:44:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 8:44:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 8:44:30 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 08:44:30.626150 31997 mini_ranger.cc:160] Time spent starting Ranger: real 9.198s user 0.000s sys 0.012s
Aug 09, 2026 8:44:51 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 8:44:51 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 8:44:51 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 8:44:51 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 8:44:52 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 08:44:52.712426 31997 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/rangerkms_127.31.63.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/rangerkms_127.31.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.63.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.63.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.63.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/HTTP_127.31.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.63.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/HTTP_127.31.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 08:44:52 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265092, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 08:44:52.799556 31997 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 08:44:52.810303 31997 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 08:44:53.721174 31997 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 08:44:53.721282 31997 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-kms
I20260809 08:44:53.721307 31997 mini_ranger_kms.cc:227] Using postgres at 127.31.63.84:60963
2026-08-09 08:44:53,932 [I] DB FLAVOR :POSTGRES
2026-08-09 08:44:53,932 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:44:53,932 [I] Checking connection
2026-08-09 08:44:53,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 08:44:54,233 [I] connection success
2026-08-09 08:44:54,233 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 08:44:54,233 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 08:44:54,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 08:44:54,503 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 08:44:54,503 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:44:54,503 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 08:44:54,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.63.84:60963/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 08:44:54,775 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 08:44:55.288089 31997 mini_ranger_kms.cc:310] Created kms service
I20260809 08:44:55.492501 31997 mini_ranger_kms.cc:326] Created kudu user
I20260809 08:44:55.536664 31997 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 08:44:55 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265092, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:44:55.932508 31997 mini_ranger_kms.cc:384] Added ranger policy
I20260809 08:44:55.932744 31997 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-kms:/tmp/dist-test-taskslFlM2/build/release/bin/postgresql.jar:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskslFlM2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 08:44:55.932770 31997 mini_ranger_kms.cc:240] Using host: 127.31.63.84
I20260809 08:44:55.939462 31997 mini_ranger_kms.cc:276] Ranger KMS PID: 17326
I20260809 08:44:55.939545 31997 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.63.84:32869
Aug 09, 2026 8:44:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:44:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:44:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 8:44:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 8:44:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 8:44:57 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 08:44:58 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265098, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.63.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 08:44:58 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265098, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.63.84@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:44:58.831584 31997 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.032s user 0.014s sys 0.177s
I20260809 08:44:58.831702 31997 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 08:44:58.831763 31997 mini_ranger_kms.cc:401] 127.31.63.84:32869/kms/v1/keys
Aug 09 08:44:59 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265098, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.63.84@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
Aug 09 08:45:01 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:45:01.584661 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33477
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33477
--ranger_config_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:32869/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:01.686067 17403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:01.686295 17403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:01.686317 17403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:01.687543 17403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:01.687597 17403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:01.687611 17403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:01.687624 17403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:01.687636 17403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:01.689270 17403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.63.84:32869/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33477
--ranger_config_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:01.689538 17403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:01.689837 17403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:45:01.695729 17403 server_base.cc:1034] running on GCE node
W20260809 08:45:01.695858 17413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:01.695712 17410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:01.695704 17411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:01.696265 17403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:01.696589 17403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:01.697781 17403 hybrid_clock.cc:648] HybridClock initialized: now 1786265101697756 us; error 41 us; skew 500 ppm
Aug 09 08:45:01 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265101, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:01.703229 17403 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:45:01.703728 17403 webserver.cc:460] Webserver started at http://127.31.63.126:34593/ using document root <none> and password file <none>
I20260809 08:45:01.703969 17403 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:01.704032 17403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:01.704206 17403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:45:01 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265101, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:45:01.755299 17403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9"
format_stamp: "Formatted at 2026-08-09 08:45:01 on dist-test-slave-gkw7"
server_key: "db56d26a58c23f09cc56b1cc590603b5"
server_key_iv: "050a0a9f89087770ee964125704117d0"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:01.755733 17403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9"
format_stamp: "Formatted at 2026-08-09 08:45:01 on dist-test-slave-gkw7"
server_key: "db56d26a58c23f09cc56b1cc590603b5"
server_key_iv: "050a0a9f89087770ee964125704117d0"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:01.757236 17403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:45:01.768929 17426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:01.769152 17403 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:01.769251 17403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9"
format_stamp: "Formatted at 2026-08-09 08:45:01 on dist-test-slave-gkw7"
server_key: "db56d26a58c23f09cc56b1cc590603b5"
server_key_iv: "050a0a9f89087770ee964125704117d0"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:01.769363 17403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:01.790330 17403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:01.792483 17403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:01.792666 17403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:01.798300 17403 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:33477
I20260809 08:45:01.798391 17498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:33477 every 8 connection(s)
I20260809 08:45:01.798745 17403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:45:01.799522 17500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:01.802050 17500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: Bootstrap starting.
I20260809 08:45:01.802695 17500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:01.803020 17500 log.cc:824] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:01.803728 17500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: No bootstrap required, opened a new log
I20260809 08:45:01.804057 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 17403
I20260809 08:45:01.804165 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:45:01.804364 31997 external_mini_cluster.cc:1428] Setting key f17cf84072e81523e67c9be6732c299f
I20260809 08:45:01.806566 17500 raft_consensus.cc:348] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } }
I20260809 08:45:01.806735 17500 raft_consensus.cc:374] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:01.806808 17500 raft_consensus.cc:729] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c13da1292cf94f9a8c9ec7f9e2f718b9, State: Initialized, Role: FOLLOWER
I20260809 08:45:01.807003 17500 consensus_queue.cc:260] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } }
I20260809 08:45:01.807121 17500 raft_consensus.cc:388] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:45:01.807183 17500 raft_consensus.cc:482] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:45:01.807258 17500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:01.807868 17500 raft_consensus.cc:504] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } }
I20260809 08:45:01.807993 17500 leader_election.cc:302] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c13da1292cf94f9a8c9ec7f9e2f718b9; no voters:
I20260809 08:45:01.808203 17500 leader_election.cc:288] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:45:01.808247 17506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:45:01.808459 17506 raft_consensus.cc:686] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 1 LEADER]: Becoming Leader. State: Replica: c13da1292cf94f9a8c9ec7f9e2f718b9, State: Running, Role: LEADER
I20260809 08:45:01.808573 17500 sys_catalog.cc:564] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:01.808601 17506 consensus_queue.cc:237] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } }
I20260809 08:45:01.809019 17506 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } } }
I20260809 08:45:01.809178 17506 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:01.809376 17506 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c13da1292cf94f9a8c9ec7f9e2f718b9. Latest consensus state: current_term: 1 leader_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } } }
I20260809 08:45:01.809505 17506 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:01.809721 17516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:45:01.810336 17516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:45:01.810636 17500 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 09 08:45:01 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:01.816356 17529 mvcc.cc:205] Tried to move back new op lower bound from 7316541857033752576 to 7316541857008590848. Current Snapshot: MvccSnapshot[applied={T|T < 7316541857033752576}]
I20260809 08:45:01.818513 17516 catalog_manager.cc:1347] Generated new cluster ID: fbb37b9e32884dd3ab845e2d47e31d92
I20260809 08:45:01.818668 17516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:45:01.819885 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:01.804903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.31.63.126:33477)
0809 08:45:01.805208 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:01.805218 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:01.805256 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:45:01.806589 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:01.806600 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:01.806674 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:01.807692 (+ 1018us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:01.807818 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:01.809270 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:01.810449 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:01.811056 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:01.813286 (+ 2230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:01.814373 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:01.814378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:01.814382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:01.814398 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:01.817248 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:01.817715 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:01.817719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:01.817719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:01.817880 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:01.818138 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:01.818139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:01.818140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:01.818602 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:01.819179 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:01.819257 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 08:45:01.819599 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":80,"threads_started":1}
I20260809 08:45:01.837833 17516 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:45:01.838673 17516 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:45:01.863277 17516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: Generated new TSK 0
I20260809 08:45:01.863600 17516 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:45:02.558809 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:32869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33477
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:02.662575 17561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:02.662835 17561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:02.662879 17561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:02.664084 17561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:02.664165 17561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:02.664212 17561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:45:02.665864 17561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.63.84:32869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:02.666145 17561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:02.666416 17561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:02.671902 17567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:02.671933 17566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:02.672091 17572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:02.672058 17561 server_base.cc:1034] running on GCE node
I20260809 08:45:02.673671 17561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:02.673977 17561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:02.675171 17561 hybrid_clock.cc:648] HybridClock initialized: now 1786265102675144 us; error 44 us; skew 500 ppm
Aug 09 08:45:02 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:02.678061 17561 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:45:02.678521 17561 webserver.cc:460] Webserver started at http://127.31.63.65:43747/ using document root <none> and password file <none>
I20260809 08:45:02.678828 17561 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:02.678906 17561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:02.679062 17561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:45:02 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:45:02.694613 17561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "e79fad49a2ca4989984dd5a92e7f6978"
format_stamp: "Formatted at 2026-08-09 08:45:02 on dist-test-slave-gkw7"
server_key: "f65338dcc29c15d155245ed6b221ea50"
server_key_iv: "862fa1751414bb872cd3e8810aa0fce5"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:02.695216 17561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "e79fad49a2ca4989984dd5a92e7f6978"
format_stamp: "Formatted at 2026-08-09 08:45:02 on dist-test-slave-gkw7"
server_key: "f65338dcc29c15d155245ed6b221ea50"
server_key_iv: "862fa1751414bb872cd3e8810aa0fce5"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:02.697453 17561 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 08:45:02.721210 17585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:02.721462 17561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 08:45:02.721588 17561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "e79fad49a2ca4989984dd5a92e7f6978"
format_stamp: "Formatted at 2026-08-09 08:45:02 on dist-test-slave-gkw7"
server_key: "f65338dcc29c15d155245ed6b221ea50"
server_key_iv: "862fa1751414bb872cd3e8810aa0fce5"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:02.721689 17561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:02.738122 17561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:02.740672 17561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:02.740846 17561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:02.741134 17561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:02.741585 17561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:02.741633 17561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:02.741703 17561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:02.741734 17561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:02.749862 17561 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:46509
I20260809 08:45:02.750305 17561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:45:02.753669 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 17561
I20260809 08:45:02.753768 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:45:02.753948 31997 external_mini_cluster.cc:1428] Setting key dc7912f6e8b63ffb7f0e74fc980bc07a
I20260809 08:45:02.751179 17740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:46509 every 8 connection(s)
Aug 09 08:45:02 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:02.763746 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:02.751814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33477 (local address 127.31.63.65:39203)
0809 08:45:02.752444 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:02.752492 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:02.753138 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:02.753642 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:02.753647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:02.753905 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:02.754252 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:02.754259 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:02.755355 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:02.755356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:02.756718 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:02.756722 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:02.757295 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:02.757296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:02.757441 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:02.758112 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:02.758124 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:02.759845 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:02.762033 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:02.762042 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:02.762045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:02.762349 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:02.762567 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:02.762567 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:02.762568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:02.762714 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:02.763019 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:02.763025 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:02.763194 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 08:45:02.763302 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:45:02.763462 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":90,"threads_started":1}
I20260809 08:45:02.764093 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:02.752132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:39203 (local address 127.31.63.126:33477)
0809 08:45:02.753352 (+ 1220us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:02.753354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:02.753363 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:45:02.753391 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:02.753392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:02.753405 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:02.753476 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:02.754371 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:02.755242 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:02.756830 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:02.757139 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:02.757209 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:02.759961 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:02.759962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:02.759963 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:02.759971 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:02.761937 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:02.762436 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:02.762437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:02.762438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:02.762501 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:02.762787 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:02.762788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:02.762788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:02.762932 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:02.762993 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:02.763902 (+ 909us) server_negotiation.cc:299] Negotiation successful
0809 08:45:02.763994 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1184,"thread_start_us":84,"threads_started":1}
I20260809 08:45:02.764765 17743 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33477
I20260809 08:45:02.764878 17743 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:02.765080 17743 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
I20260809 08:45:02.765704 17449 ts_manager.cc:194] Registered new tserver with Master: e79fad49a2ca4989984dd5a92e7f6978 (127.31.63.65:46509)
I20260809 08:45:02.766546 17449 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:39203
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:45:02.797380 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:32869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33477
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:02.921145 17754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:02.921474 17754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:02.921511 17754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:02.923417 17754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:02.923496 17754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:02.923542 17754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:45:02.926028 17754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.63.84:32869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:02.926358 17754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:02.926738 17754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:02.933809 17766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:02.934137 17764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:02.934151 17754 server_base.cc:1034] running on GCE node
W20260809 08:45:02.933836 17763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:02.937996 17754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:02.938326 17754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:02.939522 17754 hybrid_clock.cc:648] HybridClock initialized: now 1786265102939490 us; error 53 us; skew 500 ppm
Aug 09 08:45:02 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:02.943217 17754 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:45:02.943863 17754 webserver.cc:460] Webserver started at http://127.31.63.66:38459/ using document root <none> and password file <none>
I20260809 08:45:02.944198 17754 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:02.944274 17754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:02.944422 17754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:45:02 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
Aug 09 08:45:02 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:02.959173 17754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "46479a253a92457da949f9881ed432ff"
format_stamp: "Formatted at 2026-08-09 08:45:02 on dist-test-slave-gkw7"
server_key: "af8b4235dd48b88ab0a7dc76062a2538"
server_key_iv: "cc31cb2aafea0c0864cff83dc088681e"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:02.959637 17754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "46479a253a92457da949f9881ed432ff"
format_stamp: "Formatted at 2026-08-09 08:45:02 on dist-test-slave-gkw7"
server_key: "af8b4235dd48b88ab0a7dc76062a2538"
server_key_iv: "cc31cb2aafea0c0864cff83dc088681e"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:02.968227 17754 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.000s
I20260809 08:45:02.991158 17778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:02.991443 17754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 08:45:02.991547 17754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "46479a253a92457da949f9881ed432ff"
format_stamp: "Formatted at 2026-08-09 08:45:02 on dist-test-slave-gkw7"
server_key: "af8b4235dd48b88ab0a7dc76062a2538"
server_key_iv: "cc31cb2aafea0c0864cff83dc088681e"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:02.991657 17754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:03.012650 17754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:03.015044 17754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:03.015233 17754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:03.015486 17754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:03.015902 17754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:03.015959 17754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.016000 17754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:03.016026 17754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.024273 17754 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:44337
I20260809 08:45:03.024422 17939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:44337 every 8 connection(s)
I20260809 08:45:03.024816 17754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:45:03.029711 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 17754
I20260809 08:45:03.030242 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:45:03.030490 31997 external_mini_cluster.cc:1428] Setting key 85a1681ff76292a09a8df65c2c000f12
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265102, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:03.037890 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:03.025676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:44701 (local address 127.31.63.126:33477)
0809 08:45:03.025731 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:03.025733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:03.026030 (+ 297us) server_negotiation.cc:407] Connection header received
0809 08:45:03.026597 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:03.026599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:03.026620 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:03.026668 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:03.027364 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.027899 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.028941 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.029238 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.029303 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:03.034199 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:03.034201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:03.034202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:03.034210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:03.036032 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:03.036565 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:03.036566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:03.036566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:03.036641 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:03.036961 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:03.036962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:03.036963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:03.037126 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:03.037175 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:03.037635 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 08:45:03.037745 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:45:03.038448 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:03.025689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33477 (local address 127.31.63.66:44701)
0809 08:45:03.025887 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:03.025933 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:03.026471 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:03.026772 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:03.026782 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:03.026983 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:03.027266 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:03.027270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.028000 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.028001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:03.028848 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:03.028850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.029387 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.029388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:03.029559 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:03.030207 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:03.030220 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:03.034090 (+ 3870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:03.036130 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:03.036138 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:03.036141 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:03.036475 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:03.036716 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:03.036716 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:03.036717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:03.036872 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:03.037230 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:03.037239 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:03.037430 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 08:45:03.038001 (+ 571us) client_negotiation.cc:240] Negotiation successful
0809 08:45:03.038186 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260809 08:45:03.039000 17940 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33477
I20260809 08:45:03.039139 17940 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:03.039335 17940 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
I20260809 08:45:03.039767 17449 ts_manager.cc:194] Registered new tserver with Master: 46479a253a92457da949f9881ed432ff (127.31.63.66:44337)
I20260809 08:45:03.040321 17449 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:44701
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:45:03.064786 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:32869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33477
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:03.161949 17948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:03.162184 17948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:03.162225 17948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:03.163447 17948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:03.163513 17948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:03.163558 17948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:45:03.165199 17948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.63.84:32869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:03.165459 17948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:03.165766 17948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:03.172834 17954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:03.172989 17955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:03.173780 17957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:03.174580 17948 server_base.cc:1034] running on GCE node
I20260809 08:45:03.176304 17948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:03.177778 17948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:03.178960 17948 hybrid_clock.cc:648] HybridClock initialized: now 1786265103178921 us; error 49 us; skew 500 ppm
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:03.182750 17948 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:45:03.183324 17948 webserver.cc:460] Webserver started at http://127.31.63.67:41935/ using document root <none> and password file <none>
I20260809 08:45:03.183595 17948 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:03.183656 17948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:03.183794 17948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:45:03.203842 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "973621251e824adfbddda2b60c048ab7"
format_stamp: "Formatted at 2026-08-09 08:45:03 on dist-test-slave-gkw7"
server_key: "2a09793571da2586e290acd8903b27a9"
server_key_iv: "80e7cd6c67ea9b1efe2c835f185d5bc4"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:03.204319 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "973621251e824adfbddda2b60c048ab7"
format_stamp: "Formatted at 2026-08-09 08:45:03 on dist-test-slave-gkw7"
server_key: "2a09793571da2586e290acd8903b27a9"
server_key_iv: "80e7cd6c67ea9b1efe2c835f185d5bc4"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:03.210446 17948 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 08:45:03.237393 17967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.237630 17948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 08:45:03.237746 17948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "973621251e824adfbddda2b60c048ab7"
format_stamp: "Formatted at 2026-08-09 08:45:03 on dist-test-slave-gkw7"
server_key: "2a09793571da2586e290acd8903b27a9"
server_key_iv: "80e7cd6c67ea9b1efe2c835f185d5bc4"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:03.237851 17948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:03.257405 17948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:03.259666 17948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:03.259852 17948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:03.260128 17948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:03.260551 17948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:03.260589 17948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.260612 17948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:03.260653 17948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.268810 17948 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:32829
I20260809 08:45:03.268909 18127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:32829 every 8 connection(s)
I20260809 08:45:03.269249 17948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:45:03.274340 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 17948
I20260809 08:45:03.274621 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:45:03.274988 31997 external_mini_cluster.cc:1428] Setting key 0023531f5bf00facc8ba86f2ba110d83
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:03.282954 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:03.270149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:40507 (local address 127.31.63.126:33477)
0809 08:45:03.270234 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:03.270244 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:03.270526 (+ 282us) server_negotiation.cc:407] Connection header received
0809 08:45:03.271448 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:03.271449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:03.271464 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:03.271557 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:03.272639 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.273602 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.275594 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.275973 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.276059 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:03.279480 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:03.279481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:03.279482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:03.279490 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:03.281210 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:03.281801 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:03.281802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:03.281802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:03.281858 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:03.282171 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:03.282172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:03.282172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:03.282281 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:03.282333 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:03.282745 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 08:45:03.282848 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 08:45:03.283576 18134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:03.270096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33477 (local address 127.31.63.67:40507)
0809 08:45:03.270334 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:03.270402 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:03.271282 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:03.271733 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:03.271739 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:03.272069 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:03.272496 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:03.272504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.273722 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.273724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:03.275434 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:03.275439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.276253 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.276254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:03.276480 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:03.277241 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:03.277256 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:03.279367 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:03.281305 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:03.281315 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:03.281317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:03.281715 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:03.281930 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:03.281930 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:03.281931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:03.282091 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:03.282354 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:03.282362 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:03.282556 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 08:45:03.283121 (+ 565us) client_negotiation.cc:240] Negotiation successful
0809 08:45:03.283309 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":116,"threads_started":1}
I20260809 08:45:03.284082 18131 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33477
I20260809 08:45:03.284195 18131 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:03.284402 18131 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
I20260809 08:45:03.284776 17449 ts_manager.cc:194] Registered new tserver with Master: 973621251e824adfbddda2b60c048ab7 (127.31.63.67:32829)
I20260809 08:45:03.285316 17449 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:40507
I20260809 08:45:03.286983 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:45:03.287590 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 17561
I20260809 08:45:03.302080 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 17754
I20260809 08:45:03.317440 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 17948
I20260809 08:45:03.337934 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 17403
I20260809 08:45:03.353794 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33477
--webserver_interface=127.31.63.126
--webserver_port=34593
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33477
--ranger_config_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:32869/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.63.84:32869/kms with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:03.440124 18135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:03.440335 18135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:03.440356 18135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:03.441486 18135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:03.441545 18135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:03.441560 18135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:03.441572 18135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:03.441584 18135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:03.443306 18135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.63.84:32869/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33477
--ranger_config_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=34593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:03.443511 18135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:03.443681 18135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:03.448791 18145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:03.448777 18150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:03.448812 18135 server_base.cc:1034] running on GCE node
W20260809 08:45:03.448766 18147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:03.449295 18135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:03.449597 18135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:03.450754 18135 hybrid_clock.cc:648] HybridClock initialized: now 1786265103450732 us; error 41 us; skew 500 ppm
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:03.453357 18135 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:45:03.453835 18135 webserver.cc:460] Webserver started at http://127.31.63.126:34593/ using document root <none> and password file <none>
I20260809 08:45:03.454015 18135 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:03.454061 18135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:45:03.529618 18135 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:03.530521 18160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.530781 18135 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:03.530892 18135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9"
format_stamp: "Formatted at 2026-08-09 08:45:01 on dist-test-slave-gkw7"
server_key: "db56d26a58c23f09cc56b1cc590603b5"
server_key_iv: "050a0a9f89087770ee964125704117d0"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:03.531280 18135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:03.538019 18135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:03.540126 18135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:03.540289 18135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:03.545521 18135 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:33477
I20260809 08:45:03.545629 18243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:33477 every 8 connection(s)
I20260809 08:45:03.546027 18135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:45:03.547704 18244 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:45:03.548568 18244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: Bootstrap starting.
I20260809 08:45:03.550347 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 18135
I20260809 08:45:03.550665 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:32869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:46509
--local_ip_for_outbound_sockets=127.31.63.65
--tserver_master_addrs=127.31.63.126:33477
--webserver_port=43747
--webserver_interface=127.31.63.65
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.63.84:32869/kms with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:03.561247 18244 log.cc:824] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:03.563735 18244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:45:03.564080 18244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: Bootstrap complete.
I20260809 08:45:03.567793 18244 raft_consensus.cc:348] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } }
I20260809 08:45:03.568120 18244 raft_consensus.cc:729] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c13da1292cf94f9a8c9ec7f9e2f718b9, State: Initialized, Role: FOLLOWER
I20260809 08:45:03.568296 18244 consensus_queue.cc:260] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } }
I20260809 08:45:03.568361 18244 raft_consensus.cc:388] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:45:03.568403 18244 raft_consensus.cc:482] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:45:03.568445 18244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:45:03.569872 18244 raft_consensus.cc:504] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } }
I20260809 08:45:03.569983 18244 leader_election.cc:302] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c13da1292cf94f9a8c9ec7f9e2f718b9; no voters:
I20260809 08:45:03.570140 18244 leader_election.cc:288] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:45:03.570298 18251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:45:03.570411 18244 sys_catalog.cc:564] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:03.570528 18251 raft_consensus.cc:686] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [term 2 LEADER]: Becoming Leader. State: Replica: c13da1292cf94f9a8c9ec7f9e2f718b9, State: Running, Role: LEADER
I20260809 08:45:03.570663 18251 consensus_queue.cc:237] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } }
I20260809 08:45:03.570977 18251 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } } }
I20260809 08:45:03.571120 18251 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:03.571314 18251 sys_catalog.cc:455] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c13da1292cf94f9a8c9ec7f9e2f718b9. Latest consensus state: current_term: 2 leader_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c13da1292cf94f9a8c9ec7f9e2f718b9" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33477 } } }
I20260809 08:45:03.571436 18251 sys_catalog.cc:458] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9 [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:03.571601 18262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:45:03.571940 18244 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 08:45:03.572178 18262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:45:03.572356 18262 catalog_manager.cc:1259] Loaded cluster ID: fbb37b9e32884dd3ab845e2d47e31d92
I20260809 08:45:03.572422 18262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:45:03.577412 18262 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:45:03.577951 18262 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: Loaded TSK: 0
I20260809 08:45:03.578261 18262 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:45:03.644666 18247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:03.644896 18247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:03.644937 18247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:03.646258 18247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:03.646322 18247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:03.646368 18247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:45:03.647961 18247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.63.84:32869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:46509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=43747
--tserver_master_addrs=127.31.63.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:03.648219 18247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:03.648454 18247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:03.653800 18291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:03.653806 18294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:03.654174 18289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:03.655368 18247 server_base.cc:1034] running on GCE node
I20260809 08:45:03.655568 18247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:03.655818 18247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:03.659062 18247 hybrid_clock.cc:648] HybridClock initialized: now 1786265103658744 us; error 332 us; skew 500 ppm
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:03.661835 18247 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:45:03.662262 18247 webserver.cc:460] Webserver started at http://127.31.63.65:43747/ using document root <none> and password file <none>
I20260809 08:45:03.662585 18247 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:03.662674 18247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:45:03.680526 18247 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 08:45:03.681339 18318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.688917 18247 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.000s
I20260809 08:45:03.689043 18247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "e79fad49a2ca4989984dd5a92e7f6978"
format_stamp: "Formatted at 2026-08-09 08:45:02 on dist-test-slave-gkw7"
server_key: "f65338dcc29c15d155245ed6b221ea50"
server_key_iv: "862fa1751414bb872cd3e8810aa0fce5"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:03.690378 18247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:03.698753 18247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:03.700953 18247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:03.701128 18247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:03.701383 18247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:03.701826 18247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:03.701881 18247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.701921 18247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:03.701946 18247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.709082 18247 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:46509
I20260809 08:45:03.709427 18488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:46509 every 8 connection(s)
I20260809 08:45:03.710796 18247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:45:03.717754 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 18247
I20260809 08:45:03.718122 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:32869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:44337
--local_ip_for_outbound_sockets=127.31.63.66
--tserver_master_addrs=127.31.63.126:33477
--webserver_port=38459
--webserver_interface=127.31.63.66
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.63.84:32869/kms with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:03.728394 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:03.711627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33477 (local address 127.31.63.65:52991)
0809 08:45:03.711938 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:03.711975 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:03.712448 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:03.712985 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:03.712990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:03.713217 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:03.713413 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:03.713417 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.714901 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.714903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:03.716077 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:03.716080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.716916 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.716917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:03.717048 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:03.720113 (+ 3065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:03.720124 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:03.723795 (+ 3671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:03.726372 (+ 2577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:03.726379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:03.726381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:03.726712 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:03.726986 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:03.726987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:03.726988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:03.727134 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:03.727620 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:03.727627 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:03.727788 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 08:45:03.727895 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 08:45:03.728076 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":81,"threads_started":1}
I20260809 08:45:03.728941 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:03.711734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:52991 (local address 127.31.63.126:33477)
0809 08:45:03.712621 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:03.712628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:03.712655 (+ 27us) server_negotiation.cc:407] Connection header received
0809 08:45:03.712768 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:03.712771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:03.712810 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:03.712946 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:03.713573 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.714782 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.716185 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.716660 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.716764 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:03.723949 (+ 7185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:03.723954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:03.723954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:03.723968 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:03.726270 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:03.726807 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:03.726811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:03.726811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:03.726918 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:03.727206 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:03.727207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:03.727208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:03.727513 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:03.727590 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:03.728604 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0809 08:45:03.728739 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":836,"thread_start_us":89,"threads_started":1}
I20260809 08:45:03.729746 18489 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33477
I20260809 08:45:03.729852 18489 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:03.730103 18489 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
W20260809 08:45:03.731041 18489 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:33477 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 08:45:03.731134 18489 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:03.731207 18489 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
I20260809 08:45:03.731563 18489 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:03.731650 18489 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
W20260809 08:45:03.732461 18489 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:45:03.848439 18495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:03.848745 18495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:03.848788 18495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:03.850728 18495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:03.850781 18495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:03.850819 18495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:45:03.852972 18495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.63.84:32869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:44337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=38459
--tserver_master_addrs=127.31.63.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:03.853233 18495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:03.853516 18495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:03.859163 18501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:03.859236 18507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:03.859444 18503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:03.859540 18495 server_base.cc:1034] running on GCE node
I20260809 08:45:03.859920 18495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:03.860214 18495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:03.861370 18495 hybrid_clock.cc:648] HybridClock initialized: now 1786265103861345 us; error 43 us; skew 500 ppm
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:03.864256 18495 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:45:03.864786 18495 webserver.cc:460] Webserver started at http://127.31.63.66:38459/ using document root <none> and password file <none>
I20260809 08:45:03.865064 18495 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:03.865159 18495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:45:03.902068 18495 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260809 08:45:03.912091 18518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.913193 18495 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260809 08:45:03.913326 18495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "46479a253a92457da949f9881ed432ff"
format_stamp: "Formatted at 2026-08-09 08:45:02 on dist-test-slave-gkw7"
server_key: "af8b4235dd48b88ab0a7dc76062a2538"
server_key_iv: "cc31cb2aafea0c0864cff83dc088681e"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:03.913875 18495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:03.952351 18495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:03.954603 18495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:03.954800 18495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:03.955065 18495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:03.955482 18495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:03.955539 18495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.955582 18495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:03.955610 18495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:03.963183 18495 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:44337
I20260809 08:45:03.963303 18683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:44337 every 8 connection(s)
I20260809 08:45:03.963680 18495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:45:03 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265103, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:03.972445 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 18495
I20260809 08:45:03.972764 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.63.84:32869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:32829
--local_ip_for_outbound_sockets=127.31.63.67
--tserver_master_addrs=127.31.63.126:33477
--webserver_port=41935
--webserver_interface=127.31.63.67
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.63.84:32869/kms with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:03.976689 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:03.964508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:35245 (local address 127.31.63.126:33477)
0809 08:45:03.964559 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:03.964561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:03.964815 (+ 254us) server_negotiation.cc:407] Connection header received
0809 08:45:03.965554 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:03.965556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:03.965574 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:03.965658 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:03.966469 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.967289 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.968792 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.968989 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.969049 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:03.971841 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:03.971843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:03.971843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:03.971852 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:03.974846 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:03.975393 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:03.975395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:03.975395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:03.975472 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:03.975815 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:03.975817 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:03.975817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:03.975974 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:03.976060 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:03.976433 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 08:45:03.976533 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:45:03.976913 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:03.964516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33477 (local address 127.31.63.66:35245)
0809 08:45:03.964687 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:03.964731 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:03.965400 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:03.965749 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:03.965754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:03.966010 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:03.966346 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:03.966353 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.967383 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.967385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:03.968696 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:03.968699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:03.969111 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:03.969112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:03.969260 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:03.969901 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:03.969913 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:03.971732 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:03.974934 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:03.974943 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:03.974945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:03.975285 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:03.975563 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:03.975564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:03.975564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:03.975731 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:03.976067 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:03.976076 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:03.976260 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 08:45:03.976394 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 08:45:03.976576 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260809 08:45:03.977391 18684 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33477
I20260809 08:45:03.977501 18684 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:03.977780 18684 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
W20260809 08:45:03.978439 18684 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:33477 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 08:45:03.978513 18684 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:03.978576 18684 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
I20260809 08:45:03.978896 18684 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:03.978982 18684 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
W20260809 08:45:03.979270 18684 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:45:04.084309 18687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:04.084604 18687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:04.084645 18687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:04.086408 18687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:04.086472 18687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:04.086513 18687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:45:04.088766 18687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.63.84:32869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:32829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=41935
--tserver_master_addrs=127.31.63.126:33477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:04.089048 18687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:04.089331 18687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:04.096377 18699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:04.096386 18701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:04.096799 18687 server_base.cc:1034] running on GCE node
W20260809 08:45:04.097695 18698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:04.097955 18687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:04.098232 18687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:04.105683 18687 hybrid_clock.cc:648] HybridClock initialized: now 1786265104105609 us; error 58 us; skew 500 ppm
Aug 09 08:45:04 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265104, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:04.109524 18687 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:45:04.110263 18687 webserver.cc:460] Webserver started at http://127.31.63.67:41935/ using document root <none> and password file <none>
I20260809 08:45:04.110591 18687 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:04.110682 18687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:45:04 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265104, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:45:04.134732 18687 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.001s sys 0.000s
I20260809 08:45:04.135938 18713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:04.136189 18687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:04.136299 18687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "973621251e824adfbddda2b60c048ab7"
format_stamp: "Formatted at 2026-08-09 08:45:03 on dist-test-slave-gkw7"
server_key: "2a09793571da2586e290acd8903b27a9"
server_key_iv: "80e7cd6c67ea9b1efe2c835f185d5bc4"
server_key_version: "kuduclusterkey@0"
I20260809 08:45:04.136839 18687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:04.161978 18687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:04.164880 18687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:04.165061 18687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:04.165361 18687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:04.165823 18687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:04.165880 18687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:04.165913 18687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:04.165938 18687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:04.174260 18687 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:32829
I20260809 08:45:04.174342 18882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:32829 every 8 connection(s)
I20260809 08:45:04.174793 18687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:45:04.181658 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 18687
Aug 09 08:45:04 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265104, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:04.189596 18494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:04.178265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:59801 (local address 127.31.63.126:33477)
0809 08:45:04.178352 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:04.178354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:04.178842 (+ 488us) server_negotiation.cc:407] Connection header received
0809 08:45:04.179132 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:04.179134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:04.179147 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:04.179235 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:04.179905 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:04.180679 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:04.182308 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:04.182603 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:04.182686 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:04.185776 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:04.185778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:04.185778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:04.185787 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:04.187646 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:04.188200 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:04.188202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:04.188202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:04.188267 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:04.188700 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:04.188702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:04.188702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:04.188892 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:04.188944 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:04.189365 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 08:45:04.189461 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 08:45:04.190191 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:04.178227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33477 (local address 127.31.63.67:59801)
0809 08:45:04.178429 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:04.178478 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:04.179021 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:04.179360 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:04.179364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:04.179565 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:04.179789 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:04.179793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:04.180790 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:04.180791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:04.182062 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:04.182066 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:04.182723 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:04.182724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:04.182891 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:04.183539 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:04.183550 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:04.185646 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:04.187740 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:04.187749 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:04.187752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:04.188092 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:04.188347 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:04.188348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:04.188348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:04.188592 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:04.189004 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:04.189012 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:04.189194 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 08:45:04.189757 (+ 563us) client_negotiation.cc:240] Negotiation successful
0809 08:45:04.189943 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260809 08:45:04.190822 18883 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33477
I20260809 08:45:04.190928 18883 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:04.191138 18883 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
W20260809 08:45:04.191857 18883 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:33477 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 08:45:04.191937 18883 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:04.192003 18883 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
I20260809 08:45:04.192349 18883 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:04.192456 18883 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
W20260809 08:45:04.192787 18883 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:45:04.193346 18889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:04.182165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60714 (local address 127.31.63.126:33477)
0809 08:45:04.182476 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:04.182478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:04.182483 (+ 5us) server_negotiation.cc:407] Connection header received
0809 08:45:04.182501 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:04.182502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:04.182511 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:04.182592 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:04.183805 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:04.184550 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:04.185365 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:04.185721 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:04.185793 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:04.187507 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:04.187509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:04.187510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:04.187518 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:04.189265 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:04.191408 (+ 2143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:04.191409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:04.191410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:04.191478 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:04.192139 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:04.192140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:04.192140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:04.192278 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:04.192734 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:04.193120 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 08:45:04.193220 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":97,"threads_started":1}
Aug 09 08:45:04 dist-test-slave-gkw7 krb5kdc[16766](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265104, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:04.732661 18489 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:04.732841 18489 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
I20260809 08:45:04.733266 18192 ts_manager.cc:194] Registered new tserver with Master: e79fad49a2ca4989984dd5a92e7f6978 (127.31.63.65:46509)
I20260809 08:45:04.734356 18192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:52991
I20260809 08:45:04.979480 18684 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:04.979655 18684 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
I20260809 08:45:04.980140 18192 ts_manager.cc:194] Registered new tserver with Master: 46479a253a92457da949f9881ed432ff (127.31.63.66:44337)
I20260809 08:45:04.980690 18192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:35245
Aug 09 08:45:05 dist-test-slave-gkw7 krb5kdc[16766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265104, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for HTTP/127.31.63.84@KRBTEST.COM
I20260809 08:45:05.192947 18883 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:05.193120 18883 heartbeater.cc:511] Master 127.31.63.126:33477 requested a full tablet report, sending...
I20260809 08:45:05.193446 18192 ts_manager.cc:194] Registered new tserver with Master: 973621251e824adfbddda2b60c048ab7 (127.31.63.67:32829)
I20260809 08:45:05.194020 18192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:59801
I20260809 08:45:05.202302 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:45:05.210873 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.203349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60718 (local address 127.31.63.126:33477)
0809 08:45:05.203477 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:05.203480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:05.203524 (+ 44us) server_negotiation.cc:407] Connection header received
0809 08:45:05.203612 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:05.203612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:05.203623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:05.203684 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:05.204411 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.204930 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.206044 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.206282 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.206342 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.207829 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:05.207830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:05.207830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:05.207837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:05.209684 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:05.210035 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:05.210036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:05.210037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:05.210111 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:05.210325 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:05.210326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:05.210326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:05.210443 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:05.210556 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:05.210663 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 08:45:05.210751 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260809 08:45:05.213047 18192 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:45:05.214905 18192 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:45:05.223469 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.218223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:32829 (local address 127.0.0.1:43730)
0809 08:45:05.218982 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:05.218991 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:05.219036 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:05.219266 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:05.219271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:05.219274 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:05.219357 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:05.219358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.220629 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.220630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:05.222133 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:05.222135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.223166 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.223167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:05.223254 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.223267 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 08:45:05.223334 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:45:05.223386 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":57,"thread_start_us":119,"threads_started":1}
I20260809 08:45:05.223924 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.218525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43730 (local address 127.31.63.67:32829)
0809 08:45:05.218670 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:05.218675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:05.219045 (+ 370us) server_negotiation.cc:407] Connection header received
0809 08:45:05.219142 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:05.219145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:05.219162 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:05.219244 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:05.219430 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.220528 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.222241 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.222977 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.223050 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.223111 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:05.223795 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 08:45:05.223846 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":57,"threads_started":1}
I20260809 08:45:05.223945 18933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.218524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33506 (local address 127.31.63.66:44337)
0809 08:45:05.218685 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:05.218689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:05.219099 (+ 410us) server_negotiation.cc:407] Connection header received
0809 08:45:05.219146 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:05.219148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:05.219162 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:05.219244 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:05.219565 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.220593 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.222231 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.222998 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.223082 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.223794 (+ 712us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:05.223824 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 08:45:05.223870 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":54,"threads_started":1}
I20260809 08:45:05.223696 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.218223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:44337 (local address 127.0.0.1:33506)
0809 08:45:05.218740 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:05.218780 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:05.218848 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:05.219453 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:05.219454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:05.219456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:05.219490 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:05.219490 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.220662 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.220663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:05.222131 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:05.222135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.223457 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.223458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:05.223532 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.223535 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:45:05.223575 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 08:45:05.223615 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":62,"thread_start_us":117,"threads_started":1}
I20260809 08:45:05.225150 18779 tablet_service.cc:1467] Processing CreateTablet for tablet 28f8c8eea302403aba1ae35e40a64136 (DEFAULT_TABLE table=test-table [id=96eef4ba642346e5bc5a6d1b3c17f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:05.225644 18779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28f8c8eea302403aba1ae35e40a64136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:05.229856 18937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.218542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34144 (local address 127.31.63.65:46509)
0809 08:45:05.218734 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:05.218740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:05.224968 (+ 6228us) server_negotiation.cc:407] Connection header received
0809 08:45:05.225015 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:05.225018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:05.225036 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:05.225122 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:05.225343 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.226849 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.228152 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.228927 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.229010 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.229097 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:05.229725 (+ 628us) server_negotiation.cc:299] Negotiation successful
0809 08:45:05.229771 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260809 08:45:05.229753 18590 tablet_service.cc:1467] Processing CreateTablet for tablet 28f8c8eea302403aba1ae35e40a64136 (DEFAULT_TABLE table=test-table [id=96eef4ba642346e5bc5a6d1b3c17f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:05.230037 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.218231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:46509 (local address 127.0.0.1:34144)
0809 08:45:05.224834 (+ 6603us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:05.224848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:05.224896 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:05.225231 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:05.225232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:05.225234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:05.225268 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:05.225269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.226946 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.226947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:05.228031 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:05.228034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.229141 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.229143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:05.229208 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.229212 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:05.229268 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 08:45:05.229309 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6589,"thread_start_us":113,"threads_started":1}
I20260809 08:45:05.230082 18590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28f8c8eea302403aba1ae35e40a64136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:05.230706 18389 tablet_service.cc:1467] Processing CreateTablet for tablet 28f8c8eea302403aba1ae35e40a64136 (DEFAULT_TABLE table=test-table [id=96eef4ba642346e5bc5a6d1b3c17f825]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:05.231034 18389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28f8c8eea302403aba1ae35e40a64136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:05.234861 18940 tablet_bootstrap.cc:492] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978: Bootstrap starting.
I20260809 08:45:05.235067 18939 tablet_bootstrap.cc:492] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: Bootstrap starting.
I20260809 08:45:05.235638 18940 tablet_bootstrap.cc:654] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:05.235754 18939 tablet_bootstrap.cc:654] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:05.236058 18940 log.cc:824] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:05.236083 18939 log.cc:824] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:05.236460 18938 tablet_bootstrap.cc:492] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: Bootstrap starting.
I20260809 08:45:05.236972 18940 tablet_bootstrap.cc:492] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978: No bootstrap required, opened a new log
I20260809 08:45:05.237035 18940 ts_tablet_manager.cc:1397] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:45:05.237077 18938 tablet_bootstrap.cc:654] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:05.237365 18938 log.cc:824] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:05.238150 18939 tablet_bootstrap.cc:492] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: No bootstrap required, opened a new log
I20260809 08:45:05.238220 18939 ts_tablet_manager.cc:1397] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 08:45:05.239477 18940 raft_consensus.cc:348] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 } } peers { permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 } } peers { permanent_uuid: "973621251e824adfbddda2b60c048ab7" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 32829 } }
I20260809 08:45:05.239632 18940 raft_consensus.cc:374] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:05.239663 18940 raft_consensus.cc:729] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e79fad49a2ca4989984dd5a92e7f6978, State: Initialized, Role: FOLLOWER
I20260809 08:45:05.239771 18940 consensus_queue.cc:260] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 } } peers { permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 } } peers { permanent_uuid: "973621251e824adfbddda2b60c048ab7" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 32829 } }
I20260809 08:45:05.240005 18940 ts_tablet_manager.cc:1428] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:45:05.240309 18489 heartbeater.cc:503] Master 127.31.63.126:33477 was elected leader, sending a full tablet report...
I20260809 08:45:05.241112 18939 raft_consensus.cc:348] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 } } peers { permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 } } peers { permanent_uuid: "973621251e824adfbddda2b60c048ab7" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 32829 } }
I20260809 08:45:05.241273 18939 raft_consensus.cc:374] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:05.241308 18939 raft_consensus.cc:729] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46479a253a92457da949f9881ed432ff, State: Initialized, Role: FOLLOWER
I20260809 08:45:05.241479 18939 consensus_queue.cc:260] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 } } peers { permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 } } peers { permanent_uuid: "973621251e824adfbddda2b60c048ab7" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 32829 } }
I20260809 08:45:05.241765 18939 ts_tablet_manager.cc:1428] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 08:45:05.241915 18684 heartbeater.cc:503] Master 127.31.63.126:33477 was elected leader, sending a full tablet report...
I20260809 08:45:05.242816 18938 tablet_bootstrap.cc:492] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: No bootstrap required, opened a new log
I20260809 08:45:05.242908 18938 ts_tablet_manager.cc:1397] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260809 08:45:05.245244 18938 raft_consensus.cc:348] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 } } peers { permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 } } peers { permanent_uuid: "973621251e824adfbddda2b60c048ab7" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 32829 } }
I20260809 08:45:05.245414 18938 raft_consensus.cc:374] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:05.245450 18938 raft_consensus.cc:729] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 973621251e824adfbddda2b60c048ab7, State: Initialized, Role: FOLLOWER
I20260809 08:45:05.245637 18938 consensus_queue.cc:260] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 } } peers { permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 } } peers { permanent_uuid: "973621251e824adfbddda2b60c048ab7" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 32829 } }
I20260809 08:45:05.245931 18938 ts_tablet_manager.cc:1428] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:45:05.245967 18883 heartbeater.cc:503] Master 127.31.63.126:33477 was elected leader, sending a full tablet report...
I20260809 08:45:05.316011 18953 raft_consensus.cc:482] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:05.316192 18953 raft_consensus.cc:504] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 } } peers { permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 } } peers { permanent_uuid: "973621251e824adfbddda2b60c048ab7" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 32829 } }
I20260809 08:45:05.318872 18953 leader_election.cc:288] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e79fad49a2ca4989984dd5a92e7f6978 (127.31.63.65:46509), 46479a253a92457da949f9881ed432ff (127.31.63.66:44337)
I20260809 08:45:05.321305 18933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.317080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:45489 (local address 127.31.63.66:44337)
0809 08:45:05.317221 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:05.317223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:05.317233 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:45:05.317265 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:05.317266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:05.317277 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:05.317339 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:05.317809 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.318572 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.320271 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.320820 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.320905 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.320916 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:05.321174 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 08:45:05.321221 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260809 08:45:05.321623 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.316801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:44337 (local address 127.31.63.67:45489)
0809 08:45:05.317106 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:05.317120 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:05.317183 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:05.317588 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:05.317590 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:05.317593 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:05.317682 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:05.317685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.318733 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.318734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:05.320164 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:05.320168 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.320976 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.320977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:05.321049 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.321055 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:45:05.321434 (+ 379us) client_negotiation.cc:240] Negotiation successful
0809 08:45:05.321487 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":47,"thread_start_us":137,"threads_started":1}
I20260809 08:45:05.322047 18621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28f8c8eea302403aba1ae35e40a64136" candidate_uuid: "973621251e824adfbddda2b60c048ab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46479a253a92457da949f9881ed432ff" is_pre_election: true
I20260809 08:45:05.322237 18621 raft_consensus.cc:2454] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 973621251e824adfbddda2b60c048ab7 in term 0.
I20260809 08:45:05.322805 18715 leader_election.cc:302] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46479a253a92457da949f9881ed432ff, 973621251e824adfbddda2b60c048ab7; no voters:
I20260809 08:45:05.322867 18937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.317171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:52371 (local address 127.31.63.65:46509)
0809 08:45:05.317399 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:05.317401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:05.317410 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:45:05.317435 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:05.317436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:05.317444 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:05.317500 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:05.317796 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.318511 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.321297 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.321867 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.321917 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.321925 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:05.322741 (+ 816us) server_negotiation.cc:299] Negotiation successful
0809 08:45:05.322797 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
I20260809 08:45:05.322954 18953 raft_consensus.cc:2781] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:45:05.323143 18953 raft_consensus.cc:482] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:45:05.323237 18953 raft_consensus.cc:3037] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:05.323370 18955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.316808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:46509 (local address 127.31.63.67:52371)
0809 08:45:05.317261 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:05.317269 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:05.317302 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:05.317585 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:05.317586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:05.317588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:05.317697 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:05.317699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.318698 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.318699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:05.321191 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:05.321194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.322564 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.322565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:05.322647 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.322651 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:05.323068 (+ 417us) client_negotiation.cc:240] Negotiation successful
0809 08:45:05.323127 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":26,"thread_start_us":139,"threads_started":1}
I20260809 08:45:05.323829 18421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28f8c8eea302403aba1ae35e40a64136" candidate_uuid: "973621251e824adfbddda2b60c048ab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79fad49a2ca4989984dd5a92e7f6978" is_pre_election: true
I20260809 08:45:05.323988 18421 raft_consensus.cc:2454] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 973621251e824adfbddda2b60c048ab7 in term 0.
I20260809 08:45:05.324450 18953 raft_consensus.cc:504] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 } } peers { permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 } } peers { permanent_uuid: "973621251e824adfbddda2b60c048ab7" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 32829 } }
I20260809 08:45:05.324728 18421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28f8c8eea302403aba1ae35e40a64136" candidate_uuid: "973621251e824adfbddda2b60c048ab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79fad49a2ca4989984dd5a92e7f6978"
I20260809 08:45:05.324806 18421 raft_consensus.cc:3037] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:05.324812 18621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28f8c8eea302403aba1ae35e40a64136" candidate_uuid: "973621251e824adfbddda2b60c048ab7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46479a253a92457da949f9881ed432ff"
I20260809 08:45:05.324889 18621 raft_consensus.cc:3037] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:05.325835 18421 raft_consensus.cc:2454] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 973621251e824adfbddda2b60c048ab7 in term 1.
I20260809 08:45:05.325838 18621 raft_consensus.cc:2454] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 973621251e824adfbddda2b60c048ab7 in term 1.
I20260809 08:45:05.326020 18715 leader_election.cc:302] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46479a253a92457da949f9881ed432ff, 973621251e824adfbddda2b60c048ab7; no voters:
I20260809 08:45:05.326265 18956 raft_consensus.cc:2781] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:45:05.326347 18953 leader_election.cc:288] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [CANDIDATE]: Term 1 election: Requested vote from peers e79fad49a2ca4989984dd5a92e7f6978 (127.31.63.65:46509), 46479a253a92457da949f9881ed432ff (127.31.63.66:44337)
I20260809 08:45:05.326508 18956 raft_consensus.cc:686] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 1 LEADER]: Becoming Leader. State: Replica: 973621251e824adfbddda2b60c048ab7, State: Running, Role: LEADER
I20260809 08:45:05.326673 18956 consensus_queue.cc:237] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 } } peers { permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 } } peers { permanent_uuid: "973621251e824adfbddda2b60c048ab7" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 32829 } }
I20260809 08:45:05.327600 18191 catalog_manager.cc:5495] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 973621251e824adfbddda2b60c048ab7 (127.31.63.67). New cstate: current_term: 1 leader_uuid: "973621251e824adfbddda2b60c048ab7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "973621251e824adfbddda2b60c048ab7" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 32829 } health_report { overall_health: HEALTHY } } }
I20260809 08:45:05.341468 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.338047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43746 (local address 127.31.63.67:32829)
0809 08:45:05.338098 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:05.338100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:05.338107 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:45:05.338151 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:05.338152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:05.338174 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:05.338240 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:45:05.338419 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.339185 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.340325 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:05.340633 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:05.340691 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:05.340893 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:45:05.341029 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:45:05.341101 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:05.341365 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 08:45:05.341387 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:45:05.343796 18621 raft_consensus.cc:1261] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff [term 1 FOLLOWER]: Refusing update from remote peer 973621251e824adfbddda2b60c048ab7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:05.344210 18957 consensus_queue.cc:1035] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46479a253a92457da949f9881ed432ff" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 44337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:05.344379 18421 raft_consensus.cc:1261] T 28f8c8eea302403aba1ae35e40a64136 P e79fad49a2ca4989984dd5a92e7f6978 [term 1 FOLLOWER]: Refusing update from remote peer 973621251e824adfbddda2b60c048ab7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:05.344681 18957 consensus_queue.cc:1035] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e79fad49a2ca4989984dd5a92e7f6978" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 46509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:05.347303 18962 mvcc.cc:205] Tried to move back new op lower bound from 7316541871484936192 to 7316541871418933248. Current Snapshot: MvccSnapshot[applied={T|T < 7316541871484936192}]
I20260809 08:45:05.349568 18967 mvcc.cc:205] Tried to move back new op lower bound from 7316541871484936192 to 7316541871418933248. Current Snapshot: MvccSnapshot[applied={T|T < 7316541871484936192 or (T in {7316541871484936192})}]
I20260809 08:45:05.352108 18192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:45:05.352205 18192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:45:05.353056 18192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c13da1292cf94f9a8c9ec7f9e2f718b9: Sending DeleteTablet for 3 replicas of tablet 28f8c8eea302403aba1ae35e40a64136
I20260809 08:45:05.353461 18590 tablet_service.cc:1514] Processing DeleteTablet for tablet 28f8c8eea302403aba1ae35e40a64136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:05 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:33506
I20260809 08:45:05.353474 18779 tablet_service.cc:1514] Processing DeleteTablet for tablet 28f8c8eea302403aba1ae35e40a64136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:05 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:43730
I20260809 08:45:05.353904 18389 tablet_service.cc:1514] Processing DeleteTablet for tablet 28f8c8eea302403aba1ae35e40a64136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:05 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:34144
I20260809 08:45:05.354014 18963 mvcc.cc:205] Tried to move back new op lower bound from 7316541871484936192 to 7316541871418933248. Current Snapshot: MvccSnapshot[applied={T|T < 7316541871484936192}]
I20260809 08:45:05.354331 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 18247
I20260809 08:45:05.362419 18976 tablet_replica.cc:330] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: stopping tablet replica
I20260809 08:45:05.362568 18976 raft_consensus.cc:2229] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:45:05.362731 18976 raft_consensus.cc:2258] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:05.366983 18976 ts_tablet_manager.cc:1905] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:05.368839 18976 ts_tablet_manager.cc:1918] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:05.368920 18976 log.cc:1196] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/28f8c8eea302403aba1ae35e40a64136
I20260809 08:45:05.369232 18976 ts_tablet_manager.cc:1939] T 28f8c8eea302403aba1ae35e40a64136 P 46479a253a92457da949f9881ed432ff: Deleting consensus metadata
I20260809 08:45:05.369743 18163 catalog_manager.cc:4841] TS 46479a253a92457da949f9881ed432ff (127.31.63.66:44337): tablet 28f8c8eea302403aba1ae35e40a64136 (table test-table [id=96eef4ba642346e5bc5a6d1b3c17f825]) successfully deleted
I20260809 08:45:05.375279 18973 tablet_replica.cc:330] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: stopping tablet replica
I20260809 08:45:05.375440 18973 raft_consensus.cc:2229] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:45:05.375618 18973 raft_consensus.cc:2258] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:05.376211 18973 ts_tablet_manager.cc:1905] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:05.378190 18973 ts_tablet_manager.cc:1918] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:05.378280 18973 log.cc:1196] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/28f8c8eea302403aba1ae35e40a64136
I20260809 08:45:05.378567 18973 ts_tablet_manager.cc:1939] T 28f8c8eea302403aba1ae35e40a64136 P 973621251e824adfbddda2b60c048ab7: Deleting consensus metadata
W20260809 08:45:05.384793 18168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:46509 (error 108)
I20260809 08:45:05.384864 18166 catalog_manager.cc:4841] TS 973621251e824adfbddda2b60c048ab7 (127.31.63.67:32829): tablet 28f8c8eea302403aba1ae35e40a64136 (table test-table [id=96eef4ba642346e5bc5a6d1b3c17f825]) successfully deleted
I20260809 08:45:05.385890 18930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:05.385735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:46509 (local address 127.0.0.1:34154)
0809 08:45:05.385790 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:05.385842 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:46509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 08:45:05.385896 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 18495
W20260809 08:45:05.429839 18168 catalog_manager.cc:4584] TS e79fad49a2ca4989984dd5a92e7f6978 (127.31.63.65:46509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28f8c8eea302403aba1ae35e40a64136: Network error: Client connection negotiation failed: client connection to 127.31.63.65:46509: connect: Connection refused (error 111)
I20260809 08:45:05.437063 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 18687
I20260809 08:45:05.445825 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 18135
I20260809 08:45:05.464917 31997 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 08:45:05.812391 31997 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 08:45:05.812485 31997 mini_ranger.cc:66] Stopping Ranger...
I20260809 08:45:06.167205 31997 mini_ranger.cc:68] Stopped Ranger
2026-08-09 08:45:06.167 UTC [16806] LOG: received smart shutdown request
2026-08-09 08:45:06.171 UTC [16806] LOG: background worker "logical replication launcher" (PID 16819) exited with exit code 1
2026-08-09 08:45:06.172 UTC [16810] LOG: shutting down
2026-08-09 08:45:06.351 UTC [16806] LOG: database system is shut down
2026-08-09T08:45:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50267 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:45:06 dist-test-slave-gkw7 krb5kdc[18994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:45:06 dist-test-slave-gkw7 krb5kdc[18994](info): set up 2 sockets
Aug 09 08:45:06 dist-test-slave-gkw7 krb5kdc[18994](info): commencing operation
krb5kdc: starting...
W20260809 08:45:07.697777 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:45:07 dist-test-slave-gkw7 krb5kdc[18994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:45:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:45:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:45:07.825170 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:40069
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:40069,127.31.63.125:38563,127.31.63.124:41527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:07.909286 19019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:07.909528 19019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:07.909580 19019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:07.910702 19019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:07.910770 19019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:07.910796 19019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:07.910828 19019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:07.910853 19019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:07.912344 19019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:40069,127.31.63.125:38563,127.31.63.124:41527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:40069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:07.912621 19019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:07.912853 19019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:07.918131 19025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:07.918162 19029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:07.918239 19019 server_base.cc:1034] running on GCE node
W20260809 08:45:07.918114 19027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:07.918630 19019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:07.919001 19019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:07.920171 19019 hybrid_clock.cc:648] HybridClock initialized: now 1786265107920158 us; error 30 us; skew 500 ppm
Aug 09 08:45:07 dist-test-slave-gkw7 krb5kdc[18994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265107, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:07.926097 19019 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:45:07.926586 19019 webserver.cc:460] Webserver started at http://127.31.63.126:46587/ using document root <none> and password file <none>
I20260809 08:45:07.926842 19019 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:07.926914 19019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:07.927066 19019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:07.928210 19019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "f20d4c74f62d48c8841281f0557db17f"
format_stamp: "Formatted at 2026-08-09 08:45:07 on dist-test-slave-gkw7"
server_key: "1a6cae17f2d6915a45e6dc3589fe4c5b"
server_key_iv: "a3e2602e389be044fc66c281cf089953"
server_key_version: "encryptionkey@0"
I20260809 08:45:07.928598 19019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "f20d4c74f62d48c8841281f0557db17f"
format_stamp: "Formatted at 2026-08-09 08:45:07 on dist-test-slave-gkw7"
server_key: "1a6cae17f2d6915a45e6dc3589fe4c5b"
server_key_iv: "a3e2602e389be044fc66c281cf089953"
server_key_version: "encryptionkey@0"
I20260809 08:45:07.930126 19019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:45:07.931064 19045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:07.931277 19019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:07.931396 19019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "f20d4c74f62d48c8841281f0557db17f"
format_stamp: "Formatted at 2026-08-09 08:45:07 on dist-test-slave-gkw7"
server_key: "1a6cae17f2d6915a45e6dc3589fe4c5b"
server_key_iv: "a3e2602e389be044fc66c281cf089953"
server_key_version: "encryptionkey@0"
I20260809 08:45:07.931475 19019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:07.945205 19019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:07.947952 19019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:07.948148 19019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:07.952446 19019 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:40069
I20260809 08:45:07.952482 19120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:40069 every 8 connection(s)
I20260809 08:45:07.952868 19019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:45:07.953837 19122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:07.955488 19122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:45:07.961040 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 19019
I20260809 08:45:07.961199 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:45:07.961402 31997 external_mini_cluster.cc:1428] Setting key 3046843dd8fcbb706fccf61fa3d46671
Aug 09 08:45:07 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265107, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:07.966495 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:07.956307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.31.63.126:40069)
0809 08:45:07.956470 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:07.956478 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:07.956579 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:45:07.957111 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:07.957113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:07.957132 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:07.957224 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:07.957783 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:07.958649 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:07.959520 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:07.959856 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:07.959948 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:07.962411 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:07.962420 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:07.962420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:07.962427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:07.964707 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:07.965170 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:07.965172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:07.965173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:07.965258 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:07.965589 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:07.965590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:07.965591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:07.965822 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:07.965896 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:07.966153 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 08:45:07.966310 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260809 08:45:07.966614 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:07.956268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:40069 (local address 127.0.0.1:42358)
0809 08:45:07.956444 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:07.956483 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:07.956953 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:07.957254 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:07.957257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:07.957509 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:07.957698 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:07.957702 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:07.958740 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:07.958741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:07.959439 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:07.959440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:07.959939 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:07.959940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:07.960001 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:07.960566 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:07.960575 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:07.962302 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:07.964797 (+ 2495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:07.964804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:07.964806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:07.965080 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:07.965331 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:07.965332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:07.965332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:07.965448 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:07.965910 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:07.965918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:07.966005 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 08:45:07.966114 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 08:45:07.966425 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":122,"thread_start_us":94,"threads_started":1}
I20260809 08:45:07.967434 19122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:45:07.967818 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:07.967674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:38563 (local address 127.0.0.1:50702)
0809 08:45:07.967728 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:07.967784 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 08:45:07.967973 19050 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.63.125:38563: connect: Connection refused (error 111)
I20260809 08:45:07.968467 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:07.968357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:38563 (local address 127.0.0.1:50712)
0809 08:45:07.968409 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:07.968442 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 08:45:07.968613 19122 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.125:38563: Network error: Client connection negotiation failed: client connection to 127.31.63.125:38563: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.63.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab.
Entry for principal kudu/127.31.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab.
I20260809 08:45:07.988853 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.125:38563
--webserver_interface=127.31.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab
--principal=kudu/127.31.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:40069,127.31.63.125:38563,127.31.63.124:41527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:08.017892 19122 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } attempt: 1
I20260809 08:45:08.018563 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.018392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:38563 (local address 127.0.0.1:50718)
0809 08:45:08.018470 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.018520 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 08:45:08.019012 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.018910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:38563 (local address 127.0.0.1:50734)
0809 08:45:08.018960 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.018988 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 08:45:08.019171 19122 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.125:38563: Network error: Client connection negotiation failed: client connection to 127.31.63.125:38563: connect: Connection refused (error 111)
W20260809 08:45:08.075280 19130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:08.075501 19130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:08.075523 19130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:08.076588 19130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:08.076627 19130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:08.076640 19130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:08.076653 19130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:08.076665 19130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:08.078198 19130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:40069,127.31.63.125:38563,127.31.63.124:41527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab
--principal=kudu/127.31.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.125:38563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:08.078402 19130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:08.078577 19130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:08.083484 19140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:08.083498 19137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:08.083693 19130 server_base.cc:1034] running on GCE node
W20260809 08:45:08.083539 19142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:08.084033 19130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:08.084336 19130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:08.085484 19130 hybrid_clock.cc:648] HybridClock initialized: now 1786265108085474 us; error 32 us; skew 500 ppm
I20260809 08:45:08.087340 19122 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } attempt: 2
I20260809 08:45:08.087785 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.087628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:38563 (local address 127.0.0.1:50746)
0809 08:45:08.087707 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.087746 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265108, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:08.088203 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.088080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:38563 (local address 127.0.0.1:50756)
0809 08:45:08.088146 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.088175 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 08:45:08.088300 19122 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.125:38563: Network error: Client connection negotiation failed: client connection to 127.31.63.125:38563: connect: Connection refused (error 111)
I20260809 08:45:08.088436 19130 init.cc:377] Logged in from keytab as kudu/127.31.63.125@KRBTEST.COM (short username kudu)
I20260809 08:45:08.088886 19130 webserver.cc:460] Webserver started at http://127.31.63.125:37749/ using document root <none> and password file <none>
I20260809 08:45:08.089139 19130 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:08.089214 19130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:08.089375 19130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:08.090520 19130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/data/instance:
uuid: "6522924d331f48b59a50cb22576d9466"
format_stamp: "Formatted at 2026-08-09 08:45:08 on dist-test-slave-gkw7"
server_key: "85eb09b887ac23f798faece3b578edd3"
server_key_iv: "94e6c08abddd60533338a9256a93497f"
server_key_version: "encryptionkey@0"
I20260809 08:45:08.090898 19130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/wal/instance:
uuid: "6522924d331f48b59a50cb22576d9466"
format_stamp: "Formatted at 2026-08-09 08:45:08 on dist-test-slave-gkw7"
server_key: "85eb09b887ac23f798faece3b578edd3"
server_key_iv: "94e6c08abddd60533338a9256a93497f"
server_key_version: "encryptionkey@0"
I20260809 08:45:08.092257 19130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:45:08.093230 19150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:08.093456 19130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:08.093578 19130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/wal
uuid: "6522924d331f48b59a50cb22576d9466"
format_stamp: "Formatted at 2026-08-09 08:45:08 on dist-test-slave-gkw7"
server_key: "85eb09b887ac23f798faece3b578edd3"
server_key_iv: "94e6c08abddd60533338a9256a93497f"
server_key_version: "encryptionkey@0"
I20260809 08:45:08.093670 19130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:08.104143 19130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:08.106221 19130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:08.106420 19130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:08.110805 19130 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.125:38563
I20260809 08:45:08.110885 19229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.125:38563 every 8 connection(s)
I20260809 08:45:08.111251 19130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
I20260809 08:45:08.112180 19231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:08.113869 19231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:45:08.114759 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 19130
I20260809 08:45:08.114905 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-1/wal/instance
I20260809 08:45:08.115146 31997 external_mini_cluster.cc:1428] Setting key afc12392ad8609ddb2d0c6c99f52c7f9
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265108, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.63.124@KRBTEST.COM; defaulting to no policy
I20260809 08:45:08.124657 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.114667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42374 (local address 127.31.63.126:40069)
0809 08:45:08.114742 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.114746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.115039 (+ 293us) server_negotiation.cc:407] Connection header received
0809 08:45:08.115761 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.115762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.115775 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.115829 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.116823 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.117641 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.118668 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.119019 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.119100 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.121413 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.121415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.121415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.121425 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.123196 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.123616 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.123618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.123619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.123694 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.123945 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.123946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.123947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.124105 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.124190 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.124436 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.124534 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:45:08.124804 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.114650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:40069 (local address 127.0.0.1:42374)
0809 08:45:08.114880 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.114938 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:08.115654 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:08.115906 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:08.115910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:08.116386 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:08.116692 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.116699 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.117741 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.117742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.118581 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.118583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.119123 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.119124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.119245 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.119809 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:08.119818 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:08.121315 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:08.123298 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.123306 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.123308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.123532 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.123766 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.123766 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.123767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.123873 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.124184 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:08.124190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:08.124309 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 08:45:08.124392 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:45:08.124553 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260809 08:45:08.125408 19231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.124@KRBTEST.COM" created.
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265108, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
I20260809 08:45:08.133245 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.125707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:38563 (local address 127.0.0.1:50766)
0809 08:45:08.125771 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.125784 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:08.125830 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:08.126112 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:08.126112 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:08.126247 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:08.126297 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.126299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.127304 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.127305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.127988 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.127991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.128622 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.128622 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.128683 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.129110 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:08.129114 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:08.130388 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:08.132153 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.132154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.132155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.132325 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.132551 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.132551 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.132552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.132614 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.132906 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:08.132907 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:08.132948 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 08:45:08.133020 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:45:08.133124 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 08:45:08.133335 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.125789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50766 (local address 127.31.63.125:38563)
0809 08:45:08.125976 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.125980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.125997 (+ 17us) server_negotiation.cc:407] Connection header received
0809 08:45:08.126016 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.126017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.126028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.126091 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.126364 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.127220 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.128073 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.128538 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.128616 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.130484 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.130492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.130493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.130500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.132047 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.132420 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.132423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.132423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.132481 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.132676 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.132677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.132677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.132822 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.132928 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.133051 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.133198 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":160,"thread_start_us":119,"threads_started":1}
I20260809 08:45:08.133945 19231 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:45:08.134291 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.134184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45880)
0809 08:45:08.134224 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.134269 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 08:45:08.134441 19159 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
I20260809 08:45:08.134934 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.134841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45882)
0809 08:45:08.134887 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.134911 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 08:45:08.135046 19231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:41527: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab.
Entry for principal kudu/127.31.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab.
I20260809 08:45:08.145943 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.124:41527
--webserver_interface=127.31.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab
--principal=kudu/127.31.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:33785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:40069,127.31.63.125:38563,127.31.63.124:41527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:08.179461 19122 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } attempt: 3
I20260809 08:45:08.184304 19231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } attempt: 1
I20260809 08:45:08.184814 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.184664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45886)
0809 08:45:08.184728 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.184772 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 08:45:08.185438 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.185347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45898)
0809 08:45:08.185388 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.185412 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 08:45:08.185549 19231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:41527: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265107, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
I20260809 08:45:08.191874 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.179927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:38563 (local address 127.0.0.1:50778)
0809 08:45:08.180015 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.180030 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:08.180109 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:08.180302 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:08.180303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:08.180504 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:08.180576 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.180579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.181522 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.181523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.182471 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.182475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.185501 (+ 3026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.185502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.185659 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.186230 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:08.186237 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:08.188035 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:08.190442 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.190444 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.190445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.190766 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.191055 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.191057 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.191057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.191137 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.191482 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:08.191483 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:08.191541 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 08:45:08.191634 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 08:45:08.191766 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 08:45:08.191872 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.180030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50778 (local address 127.31.63.125:38563)
0809 08:45:08.180079 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.180082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.180094 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:45:08.180191 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.180192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.180215 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.180296 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.180687 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.181435 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.182581 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.182938 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.183018 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.188156 (+ 5138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.188158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.188159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.188168 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.190310 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.190881 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.190882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.190883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.190961 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.191219 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.191220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.191221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.191397 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.191491 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.191683 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.191766 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:45:08.192675 19122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:45:08.193109 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.192983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45902)
0809 08:45:08.193039 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.193071 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:45:08.193507 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.193396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45906)
0809 08:45:08.193447 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.193477 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:45:08.193621 19122 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:41527: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
I20260809 08:45:08.224781 19122 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } attempt: 1
I20260809 08:45:08.225260 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.225120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45914)
0809 08:45:08.225187 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.225228 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 08:45:08.225649 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.225568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45918)
0809 08:45:08.225594 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.225620 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260809 08:45:08.225759 19122 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:41527: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
W20260809 08:45:08.240698 19239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:08.240945 19239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:08.240984 19239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:08.242286 19239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:08.242343 19239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:08.242368 19239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:08.242398 19239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:08.242429 19239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:08.243932 19239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:33785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:40069,127.31.63.125:38563,127.31.63.124:41527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab
--principal=kudu/127.31.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.124:41527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:08.244211 19239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:08.244449 19239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:45:08.249370 19239 server_base.cc:1034] running on GCE node
W20260809 08:45:08.249352 19249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:08.249344 19251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:08.249352 19248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:08.249903 19239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:08.250236 19239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:08.251407 19239 hybrid_clock.cc:648] HybridClock initialized: now 1786265108251403 us; error 35 us; skew 500 ppm
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265108, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:08.253679 19231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } attempt: 2
I20260809 08:45:08.254015 19239 init.cc:377] Logged in from keytab as kudu/127.31.63.124@KRBTEST.COM (short username kudu)
I20260809 08:45:08.254061 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.253936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45924)
0809 08:45:08.253996 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.254026 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 08:45:08.254432 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.254343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45934)
0809 08:45:08.254379 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.254407 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260809 08:45:08.254539 19231 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:41527: Network error: Client connection negotiation failed: client connection to 127.31.63.124:41527: connect: Connection refused (error 111)
I20260809 08:45:08.254567 19239 webserver.cc:460] Webserver started at http://127.31.63.124:39079/ using document root <none> and password file <none>
I20260809 08:45:08.254787 19239 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:08.254858 19239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:08.255009 19239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:08.256052 19239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/data/instance:
uuid: "232337e93d804375b7172141afaddebe"
format_stamp: "Formatted at 2026-08-09 08:45:08 on dist-test-slave-gkw7"
server_key: "0eb0595740a00f9431365fd74efd5715"
server_key_iv: "b5ceb135952ad17c07f4dbe7144ef049"
server_key_version: "encryptionkey@0"
I20260809 08:45:08.256444 19239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/wal/instance:
uuid: "232337e93d804375b7172141afaddebe"
format_stamp: "Formatted at 2026-08-09 08:45:08 on dist-test-slave-gkw7"
server_key: "0eb0595740a00f9431365fd74efd5715"
server_key_iv: "b5ceb135952ad17c07f4dbe7144ef049"
server_key_version: "encryptionkey@0"
I20260809 08:45:08.257886 19239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:45:08.258697 19258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:08.258852 19239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:45:08.258961 19239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/wal
uuid: "232337e93d804375b7172141afaddebe"
format_stamp: "Formatted at 2026-08-09 08:45:08 on dist-test-slave-gkw7"
server_key: "0eb0595740a00f9431365fd74efd5715"
server_key_iv: "b5ceb135952ad17c07f4dbe7144ef049"
server_key_version: "encryptionkey@0"
I20260809 08:45:08.259037 19239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:08.270363 19239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:08.272204 19239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:08.272365 19239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:08.276598 19239 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.124:41527
I20260809 08:45:08.276669 19335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.124:41527 every 8 connection(s)
I20260809 08:45:08.277026 19239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
I20260809 08:45:08.278023 19336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:08.279516 19336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:45:08.282599 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 19239
I20260809 08:45:08.282723 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786264918773806-31997-0/minicluster-data/master-2/wal/instance
I20260809 08:45:08.282912 31997 external_mini_cluster.cc:1428] Setting key 249a737d6a8a25be1b1c75fd64d77d3f
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265108, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
I20260809 08:45:08.289959 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.280268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42388 (local address 127.31.63.126:40069)
0809 08:45:08.280345 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.280348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.280518 (+ 170us) server_negotiation.cc:407] Connection header received
0809 08:45:08.280988 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.280990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.281000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.281064 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.281817 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.282341 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.283747 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.283944 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.284039 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.286180 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.286182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.286183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.286216 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.288367 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.288897 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.288898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.288899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.288951 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.289245 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.289246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.289246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.289350 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.289426 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.289778 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.289866 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:45:08.290154 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.280210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:40069 (local address 127.0.0.1:42388)
0809 08:45:08.280400 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.280438 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:08.280878 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:08.281177 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:08.281180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:08.281431 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:08.281717 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.281722 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.282446 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.282447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.283631 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.283634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.284059 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.284060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.284193 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.284722 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:08.284730 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:08.286086 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:08.288454 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.288462 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.288464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.288783 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.289024 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.289025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.289026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.289174 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.289435 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:08.289443 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:08.289637 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 08:45:08.289754 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 08:45:08.289908 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260809 08:45:08.290741 19336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:45:08.292459 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.283326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.31.63.124:41527)
0809 08:45:08.283496 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.283501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.283531 (+ 30us) server_negotiation.cc:407] Connection header received
0809 08:45:08.283593 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.283596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.283618 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.283711 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.284337 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.285157 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.286059 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.286367 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.286465 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.289334 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.289341 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.289342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.289348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.291261 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.291635 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.291638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.291638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.291700 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.291890 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.291891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.291891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.292054 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.292221 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.292269 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.292347 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265108, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:08.298094 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.291030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:38563 (local address 127.0.0.1:50788)
0809 08:45:08.291096 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.291105 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:08.291144 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:08.291314 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:08.291315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:08.291468 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:08.291508 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.291510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.292391 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.292392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.293102 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.293105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.293635 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.293636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.293709 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.294049 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:08.294055 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:08.295367 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:08.297061 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.297062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.297063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.297242 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.297454 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.297455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.297455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.297508 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.297872 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:08.297872 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:08.297910 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 08:45:08.297965 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:45:08.298033 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 08:45:08.298204 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.291118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.31.63.125:38563)
0809 08:45:08.291170 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.291172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.291181 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:45:08.291209 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.291210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.291220 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.291294 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.291585 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.292304 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.293165 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.293506 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.293631 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.295463 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.295465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.295465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.295473 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.296963 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.297316 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.297318 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.297318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.297382 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.297635 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.297637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.297637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.297790 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.297868 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.298013 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.298088 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:45:08.298624 19336 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:45:08.300907 19122 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } attempt: 2
I20260809 08:45:08.301461 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.293301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.31.63.126:40069)
0809 08:45:08.293359 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.293361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.293368 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:45:08.293387 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.293388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.293397 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.293452 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.294090 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.294521 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.295458 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.295753 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.295833 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.298327 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.298328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.298328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.298336 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.300143 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.300542 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.300543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.300543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.300597 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.300797 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.300798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.300798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.300897 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.301207 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.301253 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.301332 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265108, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265107, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
I20260809 08:45:08.306593 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.298884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45938)
0809 08:45:08.298939 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.298949 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:08.298984 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:08.299126 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:08.299127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:08.299305 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:08.299342 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.299344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.300268 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.300269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.301032 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.301034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.301501 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.301502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.301590 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.301948 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:08.301953 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:08.303147 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:08.305621 (+ 2474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.305622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.305623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.305785 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.305972 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.305973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.305973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.306024 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.306271 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:08.306272 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:08.306309 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 08:45:08.306395 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:45:08.306488 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 08:45:08.306645 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.298974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.31.63.124:41527)
0809 08:45:08.299013 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.299015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.299023 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:45:08.299044 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.299045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.299056 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.299126 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.299423 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.300173 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.301115 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.301430 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.301504 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.303232 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.303233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.303233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.303240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.305521 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.305869 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.305870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.305871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.305922 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.306095 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.306096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.306096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.306205 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.306278 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.306419 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.306537 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":22}
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
I20260809 08:45:08.308161 19336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe: Bootstrap starting.
I20260809 08:45:08.309005 19336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:08.309366 19336 log.cc:824] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:08.309818 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.301127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45954)
0809 08:45:08.301170 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.301179 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:08.301210 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:08.301862 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:08.301863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:08.302028 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:08.302065 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.302067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.303113 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.303114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.303844 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.303847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.304338 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.304340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.304406 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.304784 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:08.304788 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:08.306120 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:08.308558 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.308559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.308559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.308774 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.309029 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.309030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.309030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.309099 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.309465 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:08.309466 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:08.309514 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:45:08.309604 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 08:45:08.309702 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 08:45:08.310321 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.301409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45954 (local address 127.31.63.124:41527)
0809 08:45:08.301732 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.301733 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.301742 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:45:08.301761 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.301762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.301772 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.301859 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.302175 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.303017 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.303932 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.304253 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.304316 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.306216 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.306217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.306218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.306226 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.308468 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.308874 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.308875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.308875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.308935 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.309188 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.309189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.309189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.309353 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.310019 (+ 666us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.310065 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.310161 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":95,"threads_started":1}
I20260809 08:45:08.310423 19336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe: No bootstrap required, opened a new log
I20260809 08:45:08.311573 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.302281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50798 (local address 127.31.63.125:38563)
0809 08:45:08.302318 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.302320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.302329 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:45:08.302351 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.302352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.302362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.302426 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.302990 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.303698 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.304610 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.304909 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.304967 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.308455 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.308456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.308456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.308464 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.310282 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.310690 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.310692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.310692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.310749 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.310973 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.310974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.310974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.311088 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.311308 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.311350 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.311424 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:45:08.311823 19122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f: Bootstrap starting.
I20260809 08:45:08.312523 19122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:08.312557 19336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } }
I20260809 08:45:08.312706 19336 raft_consensus.cc:374] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:08.312745 19336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 232337e93d804375b7172141afaddebe, State: Initialized, Role: FOLLOWER
I20260809 08:45:08.312850 19122 log.cc:824] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:08.312875 19336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } }
I20260809 08:45:08.313035 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:45:08.313216 19336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:08.313395 19345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } } }
I20260809 08:45:08.313467 19345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:45:08.313925 19122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f: No bootstrap required, opened a new log
W20260809 08:45:08.315339 19371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:45:08.315402 19371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:45:08.316628 19122 raft_consensus.cc:348] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } }
I20260809 08:45:08.316766 19122 raft_consensus.cc:374] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:08.316809 19122 raft_consensus.cc:729] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f20d4c74f62d48c8841281f0557db17f, State: Initialized, Role: FOLLOWER
I20260809 08:45:08.316933 19122 consensus_queue.cc:260] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } }
I20260809 08:45:08.317209 19122 sys_catalog.cc:564] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:08.315500 19239 master_runner.cc:186] Error getting master registration for 127.31.63.126:40069: OK, instance_id { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" instance_seqno: 1786265107951214 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:45:08.318670 19239 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:45:08.320142 19019 master_runner.cc:186] Error getting master registration for 127.31.63.125:38563: OK, instance_id { permanent_uuid: "6522924d331f48b59a50cb22576d9466" instance_seqno: 1786265108109462 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:45:08.323341 19019 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:45:08.323565 19376 sys_catalog.cc:455] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } } }
I20260809 08:45:08.323652 19376 sys_catalog.cc:458] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:45:08.323868 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.315304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45962 (local address 127.31.63.124:41527)
0809 08:45:08.315334 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.315336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.315868 (+ 532us) server_negotiation.cc:407] Connection header received
0809 08:45:08.315968 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.315969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.315983 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.316015 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.316906 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.317672 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.318817 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.319139 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.319206 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.320725 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.320726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.320727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.320734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.322384 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.322790 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.322792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.322792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.322850 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.323049 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.323050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.323050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.323246 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.323352 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.323690 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.323777 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 08:45:08.325520 19393 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:45:08.325625 19393 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:45:08.327626 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.315138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.31.63.126:40069)
0809 08:45:08.315260 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.315263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.315533 (+ 270us) server_negotiation.cc:407] Connection header received
0809 08:45:08.315641 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.315642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.315653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.315709 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.316449 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.317365 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.318556 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.318866 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.318916 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.324659 (+ 5743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.324660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.324660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.324668 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.326258 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.326653 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.326654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.326654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.326712 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.326904 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.326905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.326905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.327028 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.327136 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.327466 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.327541 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 08:45:08.327761 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.315227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.31.63.125:38563)
0809 08:45:08.315265 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.315267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.315720 (+ 453us) server_negotiation.cc:407] Connection header received
0809 08:45:08.315777 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.315778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.315790 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.315842 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.319249 (+ 3407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.320205 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.321422 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.322075 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.322127 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.323799 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.323801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.323801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.323808 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.326046 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.326434 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.326435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.326436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.326489 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.326696 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.326697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.326697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.326820 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.327363 (+ 543us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.327584 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.327664 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:45:08.345764 19231 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } attempt: 3
Aug 09 08:45:08 dist-test-slave-gkw7 krb5kdc[18994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265108, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
I20260809 08:45:08.354194 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.346046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:41527 (local address 127.0.0.1:45966)
0809 08:45:08.346093 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:08.346105 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:08.346186 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:08.346345 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:08.346346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:08.346528 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:08.346578 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.346580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.347554 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.347556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.348386 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:08.348389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.348889 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.348890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:08.348955 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.349460 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:08.349466 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:08.351109 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:08.353038 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.353039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.353040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.353266 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.353492 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:08.353493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:08.353493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:08.353592 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:08.353859 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:08.353860 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:08.353909 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 08:45:08.353983 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 08:45:08.354085 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 08:45:08.354205 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:08.346108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45966 (local address 127.31.63.124:41527)
0809 08:45:08.346176 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:08.346178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:08.346188 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:45:08.346254 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:08.346255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:08.346266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:08.346320 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:08.346675 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.347436 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.348484 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:08.348803 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:08.348876 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:08.351205 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:08.351207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:08.351207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:08.351214 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:08.352945 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.353348 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.353349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.353350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.353413 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:08.353651 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:08.353652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:08.353652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:08.353786 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:08.353854 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:08.354023 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 08:45:08.354106 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:45:08.356436 19231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466: Bootstrap starting.
I20260809 08:45:08.357256 19231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:08.357646 19231 log.cc:824] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:08.358604 19231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466: No bootstrap required, opened a new log
I20260809 08:45:08.360723 19231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } }
I20260809 08:45:08.360886 19231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:08.360927 19231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6522924d331f48b59a50cb22576d9466, State: Initialized, Role: FOLLOWER
I20260809 08:45:08.361048 19231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } }
I20260809 08:45:08.362082 19231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:08.362030 19397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } } }
I20260809 08:45:08.362251 19397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:45:08.364089 19414 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:45:08.364144 19414 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:45:08.366130 19130 master_runner.cc:263] No leader master found from master 6522924d331f48b59a50cb22576d9466
I20260809 08:45:08.366178 19130 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 08:45:08.449105 19055 tablet.cc:2369] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:45:08.454330 19345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:08.454459 19345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } }
I20260809 08:45:08.454692 19345 leader_election.cc:288] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f20d4c74f62d48c8841281f0557db17f (127.31.63.126:40069), 6522924d331f48b59a50cb22576d9466 (127.31.63.125:38563)
I20260809 08:45:08.455232 19083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "232337e93d804375b7172141afaddebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f20d4c74f62d48c8841281f0557db17f" is_pre_election: true
I20260809 08:45:08.455415 19083 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 232337e93d804375b7172141afaddebe in term 0.
I20260809 08:45:08.455693 19261 leader_election.cc:302] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 232337e93d804375b7172141afaddebe, f20d4c74f62d48c8841281f0557db17f; no voters:
I20260809 08:45:08.455998 19345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:45:08.456055 19345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:45:08.456076 19345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:08.457005 19345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } }
I20260809 08:45:08.457126 19345 leader_election.cc:288] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [CANDIDATE]: Term 1 election: Requested vote from peers f20d4c74f62d48c8841281f0557db17f (127.31.63.126:40069), 6522924d331f48b59a50cb22576d9466 (127.31.63.125:38563)
I20260809 08:45:08.457520 19083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "232337e93d804375b7172141afaddebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f20d4c74f62d48c8841281f0557db17f"
I20260809 08:45:08.457670 19083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:08.458508 19083 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 232337e93d804375b7172141afaddebe in term 1.
I20260809 08:45:08.458707 19261 leader_election.cc:302] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 232337e93d804375b7172141afaddebe, f20d4c74f62d48c8841281f0557db17f; no voters:
I20260809 08:45:08.459043 19191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "232337e93d804375b7172141afaddebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6522924d331f48b59a50cb22576d9466" is_pre_election: true
I20260809 08:45:08.459180 19191 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 232337e93d804375b7172141afaddebe in term 0.
I20260809 08:45:08.459263 19190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "232337e93d804375b7172141afaddebe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6522924d331f48b59a50cb22576d9466"
I20260809 08:45:08.459308 19190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:08.460065 19190 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 232337e93d804375b7172141afaddebe in term 1.
I20260809 08:45:08.460453 19345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:45:08.460639 19345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [term 1 LEADER]: Becoming Leader. State: Replica: 232337e93d804375b7172141afaddebe, State: Running, Role: LEADER
I20260809 08:45:08.460724 19345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } }
I20260809 08:45:08.461098 19345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 232337e93d804375b7172141afaddebe. Latest consensus state: current_term: 1 leader_uuid: "232337e93d804375b7172141afaddebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } } }
I20260809 08:45:08.461158 19345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:08.461505 19423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:45:08.461756 19423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:45:08.463973 19190 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [term 1 FOLLOWER]: Refusing update from remote peer 232337e93d804375b7172141afaddebe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:08.464322 19345 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:08.465682 19397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 232337e93d804375b7172141afaddebe. Latest consensus state: current_term: 1 leader_uuid: "232337e93d804375b7172141afaddebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } } }
I20260809 08:45:08.465767 19397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:45:08.466192 19083 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [term 1 FOLLOWER]: Refusing update from remote peer 232337e93d804375b7172141afaddebe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:08.466468 19345 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [LEADER]: Connected to new peer: Peer: permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:08.468462 19376 sys_catalog.cc:455] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 232337e93d804375b7172141afaddebe. Latest consensus state: current_term: 1 leader_uuid: "232337e93d804375b7172141afaddebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } } }
I20260809 08:45:08.468576 19376 sys_catalog.cc:458] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:45:08.469576 19345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "232337e93d804375b7172141afaddebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } } }
I20260809 08:45:08.469607 19425 mvcc.cc:205] Tried to move back new op lower bound from 7316541884265906176 to 7316541884255793152. Current Snapshot: MvccSnapshot[applied={T|T < 7316541884265906176}]
I20260809 08:45:08.469671 19345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:08.469964 19376 sys_catalog.cc:455] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "232337e93d804375b7172141afaddebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } } }
I20260809 08:45:08.470037 19376 sys_catalog.cc:458] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:45:08.470118 19431 mvcc.cc:205] Tried to move back new op lower bound from 7316541884265906176 to 7316541884255793152. Current Snapshot: MvccSnapshot[applied={T|T < 7316541884265906176}]
I20260809 08:45:08.470741 19345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "232337e93d804375b7172141afaddebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } } }
I20260809 08:45:08.470835 19345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:08.471199 19397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "232337e93d804375b7172141afaddebe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f20d4c74f62d48c8841281f0557db17f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40069 } } peers { permanent_uuid: "6522924d331f48b59a50cb22576d9466" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 38563 } } peers { permanent_uuid: "232337e93d804375b7172141afaddebe" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 41527 } } }
I20260809 08:45:08.471279 19397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:45:08.471160 19428 mvcc.cc:205] Tried to move back new op lower bound from 7316541884265906176 to 7316541884255793152. Current Snapshot: MvccSnapshot[applied={T|T < 7316541884265906176}]
I20260809 08:45:08.471817 19423 catalog_manager.cc:1347] Generated new cluster ID: c6b7263429f4442ba4d4d1fca70e500d
I20260809 08:45:08.471866 19423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:45:08.499573 19423 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:45:08.500416 19423 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:45:08.514086 19423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 232337e93d804375b7172141afaddebe: Generated new TSK 0
I20260809 08:45:08.514498 19423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:45:09.326094 19393 catalog_manager.cc:1259] Loaded cluster ID: c6b7263429f4442ba4d4d1fca70e500d
I20260809 08:45:09.326177 19393 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f: loading cluster ID for follower catalog manager: success
I20260809 08:45:09.326972 19393 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f: acquiring CA information for follower catalog manager: success
I20260809 08:45:09.327262 19393 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f20d4c74f62d48c8841281f0557db17f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:45:09.364653 19414 catalog_manager.cc:1259] Loaded cluster ID: c6b7263429f4442ba4d4d1fca70e500d
I20260809 08:45:09.364737 19414 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466: loading cluster ID for follower catalog manager: success
I20260809 08:45:09.365579 19414 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466: acquiring CA information for follower catalog manager: success
I20260809 08:45:09.365854 19414 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6522924d331f48b59a50cb22576d9466: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:45:09.571044 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 19019
I20260809 08:45:09.581660 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 19130
I20260809 08:45:09.593389 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 19239
2026-08-09T08:45:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3128 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:45:09 dist-test-slave-gkw7 krb5kdc[19453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:45:09 dist-test-slave-gkw7 krb5kdc[19453](info): set up 2 sockets
Aug 09 08:45:09 dist-test-slave-gkw7 krb5kdc[19453](info): commencing operation
krb5kdc: starting...
W20260809 08:45:10.821334 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:45:10 dist-test-slave-gkw7 krb5kdc[19453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:45:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:45:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:45:10.950778 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:40853
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:36353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:40853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:11.034839 19472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:11.035101 19472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:11.035142 19472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:11.036317 19472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:11.036393 19472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:11.036417 19472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:11.036437 19472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:11.036468 19472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:11.038046 19472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:36353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:40853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:40853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:11.038347 19472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:11.038601 19472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:11.043761 19481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:11.043767 19483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:11.043964 19472 server_base.cc:1034] running on GCE node
W20260809 08:45:11.043771 19480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:11.044361 19472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:11.044730 19472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:11.045933 19472 hybrid_clock.cc:648] HybridClock initialized: now 1786265111045943 us; error 55 us; skew 500 ppm
Aug 09 08:45:11 dist-test-slave-gkw7 krb5kdc[19453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265111, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:11.048781 19472 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:45:11.049371 19472 webserver.cc:460] Webserver started at http://127.31.63.126:46835/ using document root <none> and password file <none>
I20260809 08:45:11.049696 19472 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:11.049773 19472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:11.049957 19472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:11.051182 19472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "0fecd6b11734459b9e33f68b53e5d281"
format_stamp: "Formatted at 2026-08-09 08:45:11 on dist-test-slave-gkw7"
server_key: "25f7e4c38b83639917af04835533e8e6"
server_key_iv: "237690211014c33ca3445c3a03ae55cb"
server_key_version: "encryptionkey@0"
I20260809 08:45:11.051594 19472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "0fecd6b11734459b9e33f68b53e5d281"
format_stamp: "Formatted at 2026-08-09 08:45:11 on dist-test-slave-gkw7"
server_key: "25f7e4c38b83639917af04835533e8e6"
server_key_iv: "237690211014c33ca3445c3a03ae55cb"
server_key_version: "encryptionkey@0"
I20260809 08:45:11.053129 19472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:45:11.054065 19493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:11.054313 19472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:45:11.054436 19472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "0fecd6b11734459b9e33f68b53e5d281"
format_stamp: "Formatted at 2026-08-09 08:45:11 on dist-test-slave-gkw7"
server_key: "25f7e4c38b83639917af04835533e8e6"
server_key_iv: "237690211014c33ca3445c3a03ae55cb"
server_key_version: "encryptionkey@0"
I20260809 08:45:11.054522 19472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:11.073184 19472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:11.076004 19472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:11.076210 19472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:11.082687 19472 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:40853
I20260809 08:45:11.082720 19566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:40853 every 8 connection(s)
I20260809 08:45:11.083151 19472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:45:11.084087 19569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:11.086484 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 19472
I20260809 08:45:11.086618 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:45:11.086705 19569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281: Bootstrap starting.
I20260809 08:45:11.086822 31997 external_mini_cluster.cc:1428] Setting key 0fddcee9a1a949b33d852ea97f19c2cc
I20260809 08:45:11.087417 19569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:11.087750 19569 log.cc:824] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:11.088546 19569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281: No bootstrap required, opened a new log
I20260809 08:45:11.090546 19569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fecd6b11734459b9e33f68b53e5d281" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40853 } }
I20260809 08:45:11.090713 19569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:11.090764 19569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fecd6b11734459b9e33f68b53e5d281, State: Initialized, Role: FOLLOWER
I20260809 08:45:11.090909 19569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fecd6b11734459b9e33f68b53e5d281" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40853 } }
I20260809 08:45:11.091012 19569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:45:11.091058 19569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:45:11.091114 19569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:11.091678 19569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fecd6b11734459b9e33f68b53e5d281" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40853 } }
I20260809 08:45:11.091800 19569 leader_election.cc:302] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fecd6b11734459b9e33f68b53e5d281; no voters:
I20260809 08:45:11.091995 19569 leader_election.cc:288] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:45:11.092021 19579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:45:11.092175 19579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [term 1 LEADER]: Becoming Leader. State: Replica: 0fecd6b11734459b9e33f68b53e5d281, State: Running, Role: LEADER
I20260809 08:45:11.092330 19569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:11.092337 19579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fecd6b11734459b9e33f68b53e5d281" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40853 } }
I20260809 08:45:11.092751 19580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fecd6b11734459b9e33f68b53e5d281. Latest consensus state: current_term: 1 leader_uuid: "0fecd6b11734459b9e33f68b53e5d281" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fecd6b11734459b9e33f68b53e5d281" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40853 } } }
I20260809 08:45:11.092911 19580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:11.093719 19581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fecd6b11734459b9e33f68b53e5d281" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fecd6b11734459b9e33f68b53e5d281" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 40853 } } }
I20260809 08:45:11.093788 19581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281 [sys.catalog]: This master's current role is: LEADER
Aug 09 08:45:11 dist-test-slave-gkw7 krb5kdc[19453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
W20260809 08:45:11.094340 19602 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:45:11.094403 19602 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:45:11.094451 19590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:45:11.094661 19590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:45:11.096326 19590 catalog_manager.cc:1347] Generated new cluster ID: aaba711d8b104924ba9b51f2d3ce3c27
I20260809 08:45:11.096385 19590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:45:11.098582 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:11.087417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.31.63.126:40853)
0809 08:45:11.087673 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:11.087681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:11.087707 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:45:11.088355 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:11.088363 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:11.088415 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:11.088593 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:11.089274 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:11.090241 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:11.091204 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:11.091623 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:11.091748 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:11.094781 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:11.094786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:11.094787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:11.094799 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:11.096965 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:11.097363 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:11.097365 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:11.097366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:11.097457 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:11.097728 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:11.097729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:11.097729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:11.097969 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:11.098145 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:11.098228 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 08:45:11.098369 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":80,"threads_started":1}
I20260809 08:45:11.102162 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:45:11.109151 19590 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:45:11.109761 19590 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:45:11.117182 19590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fecd6b11734459b9e33f68b53e5d281: Generated new TSK 0
I20260809 08:45:11.117395 19590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:45:11.156342 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 19472
2026-08-09T08:45:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1563 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:45:11 dist-test-slave-gkw7 krb5kdc[19614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:45:11 dist-test-slave-gkw7 krb5kdc[19614](info): set up 2 sockets
Aug 09 08:45:11 dist-test-slave-gkw7 krb5kdc[19614](info): commencing operation
krb5kdc: starting...
W20260809 08:45:12.395007 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:45:12 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:45:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:45:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:45:12.523947 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37975
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:37975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:12.607429 19630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:12.607684 19630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:12.607733 19630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:12.608881 19630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:12.608954 19630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:12.608979 19630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:12.609001 19630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:12.609030 19630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:12.610580 19630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:37975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:12.610872 19630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:12.611104 19630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:12.616029 19639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:12.616055 19642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:12.616058 19638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:12.616266 19630 server_base.cc:1034] running on GCE node
I20260809 08:45:12.616477 19630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:12.616780 19630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:12.617923 19630 hybrid_clock.cc:648] HybridClock initialized: now 1786265112617914 us; error 36 us; skew 500 ppm
Aug 09 08:45:12 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265112, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:12.620474 19630 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:45:12.620949 19630 webserver.cc:460] Webserver started at http://127.31.63.126:41029/ using document root <none> and password file <none>
I20260809 08:45:12.621212 19630 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:12.621286 19630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:12.621440 19630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:12.622560 19630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "fcbad767f2244341a799a96daa358c1e"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "21459d89286fa2943a6c94b828579538"
server_key_iv: "e5ff227e3756564b194080d25e67bad6"
server_key_version: "encryptionkey@0"
I20260809 08:45:12.622937 19630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "fcbad767f2244341a799a96daa358c1e"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "21459d89286fa2943a6c94b828579538"
server_key_iv: "e5ff227e3756564b194080d25e67bad6"
server_key_version: "encryptionkey@0"
I20260809 08:45:12.624310 19630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:45:12.625104 19653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:12.625306 19630 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:12.625403 19630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "fcbad767f2244341a799a96daa358c1e"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "21459d89286fa2943a6c94b828579538"
server_key_iv: "e5ff227e3756564b194080d25e67bad6"
server_key_version: "encryptionkey@0"
I20260809 08:45:12.625487 19630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:12.644838 19630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:12.646970 19630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:12.647147 19630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:12.651705 19630 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:37975
I20260809 08:45:12.651744 19725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:37975 every 8 connection(s)
I20260809 08:45:12.652146 19630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:45:12.653189 19731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:12.655570 19731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Bootstrap starting.
I20260809 08:45:12.656141 19731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:12.656388 19731 log.cc:824] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:12.657104 19731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: No bootstrap required, opened a new log
I20260809 08:45:12.658744 19731 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:12.658882 19731 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:12.658905 19731 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcbad767f2244341a799a96daa358c1e, State: Initialized, Role: FOLLOWER
I20260809 08:45:12.659008 19731 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:12.659060 19731 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:45:12.659080 19731 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:45:12.659162 19731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:12.659600 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 19630
I20260809 08:45:12.659722 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:45:12.659965 31997 external_mini_cluster.cc:1428] Setting key 0b6fb7a3024588be1046be92027dbf12
I20260809 08:45:12.660293 19731 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:12.660422 19731 leader_election.cc:302] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcbad767f2244341a799a96daa358c1e; no voters:
I20260809 08:45:12.660637 19731 leader_election.cc:288] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:45:12.660832 19739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:45:12.660923 19731 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:12.661139 19739 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 1 LEADER]: Becoming Leader. State: Replica: fcbad767f2244341a799a96daa358c1e, State: Running, Role: LEADER
I20260809 08:45:12.661291 19739 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:12.661849 19735 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcbad767f2244341a799a96daa358c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } } }
I20260809 08:45:12.661942 19735 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:12.662137 19741 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcbad767f2244341a799a96daa358c1e. Latest consensus state: current_term: 1 leader_uuid: "fcbad767f2244341a799a96daa358c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } } }
I20260809 08:45:12.662217 19741 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: This master's current role is: LEADER
W20260809 08:45:12.663116 19764 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:45:12.663179 19764 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:45:12.663234 19749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:45:12.663398 19749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:45:12.665086 19749 catalog_manager.cc:1347] Generated new cluster ID: fa108a38c71948c2b5b6e37a394b4197
I20260809 08:45:12.665140 19749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:45:12 dist-test-slave-gkw7 krb5kdc[19614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:12.671022 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:12.660525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.31.63.126:37975)
0809 08:45:12.660764 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:12.660775 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:12.660848 (+ 73us) server_negotiation.cc:407] Connection header received
0809 08:45:12.660996 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:12.661005 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:12.661065 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:12.662175 (+ 1110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:12.662303 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:12.663529 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:12.664451 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:12.664999 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:12.665149 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:12.667641 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:12.667646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:12.667646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:12.667658 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:12.669373 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:12.669791 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:12.669794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:12.669795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:12.669903 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:12.670154 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:12.670155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:12.670156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:12.670395 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:12.670580 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:12.670675 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 08:45:12.670830 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260809 08:45:12.680548 19749 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:45:12.681088 19749 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:45:12.689208 19749 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Generated new TSK 0
I20260809 08:45:12.689405 19749 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:45:12.724311 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37975
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:12.808961 19773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:12.809181 19773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:12.809204 19773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:12.810416 19773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:12.810460 19773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:12.810487 19773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:45:12.811868 19773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:12.812049 19773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:12.812227 19773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:12.817422 19785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:12.817438 19773 server_base.cc:1034] running on GCE node
W20260809 08:45:12.817423 19781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:12.817431 19780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:12.817901 19773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:12.818135 19773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:12.819293 19773 hybrid_clock.cc:648] HybridClock initialized: now 1786265112819298 us; error 45 us; skew 500 ppm
Aug 09 08:45:12 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265112, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:12.822157 19773 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:45:12.822634 19773 webserver.cc:460] Webserver started at http://127.31.63.65:39155/ using document root <none> and password file <none>
I20260809 08:45:12.822873 19773 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:12.822918 19773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:12.823068 19773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:12.824132 19773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "5f3bd4b00fce4eb9800f05248c6e3987"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "7a9f28125836c6bf129722f2fc1496fd"
server_key_iv: "4c288b83df27453e0b2257d45681d8d7"
server_key_version: "encryptionkey@0"
I20260809 08:45:12.824501 19773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "5f3bd4b00fce4eb9800f05248c6e3987"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "7a9f28125836c6bf129722f2fc1496fd"
server_key_iv: "4c288b83df27453e0b2257d45681d8d7"
server_key_version: "encryptionkey@0"
I20260809 08:45:12.826071 19773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:45:12.826843 19797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:12.827075 19773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:12.827152 19773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "5f3bd4b00fce4eb9800f05248c6e3987"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "7a9f28125836c6bf129722f2fc1496fd"
server_key_iv: "4c288b83df27453e0b2257d45681d8d7"
server_key_version: "encryptionkey@0"
I20260809 08:45:12.827248 19773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:12.841758 19773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:12.844234 19773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:12.844408 19773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:12.844672 19773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:12.844995 19773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:12.845029 19773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:12.845072 19773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:12.845101 19773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:12.851351 19773 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:38159
I20260809 08:45:12.851413 19962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:38159 every 8 connection(s)
I20260809 08:45:12.851747 19773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:45:12 dist-test-slave-gkw7 krb5kdc[19614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265112, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:12.860414 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 19773
I20260809 08:45:12.860548 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:45:12.860744 31997 external_mini_cluster.cc:1428] Setting key 50b50238721cec9538bd08d8d63ebcd7
I20260809 08:45:12.861334 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:12.852535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:44209 (local address 127.31.63.126:37975)
0809 08:45:12.852609 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:12.852612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:12.852837 (+ 225us) server_negotiation.cc:407] Connection header received
0809 08:45:12.853406 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:12.853407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:12.853423 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:12.853485 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:12.854525 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:12.855059 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:12.856068 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:12.856293 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:12.856350 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:12.858498 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:12.858499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:12.858500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:12.858508 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:12.859796 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:12.860243 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:12.860244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:12.860245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:12.860307 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:12.860592 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:12.860593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:12.860593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:12.860740 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:12.860815 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:12.861152 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 08:45:12.861245 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:45:12.861519 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:12.852491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:44209)
0809 08:45:12.852725 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:12.852764 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:12.853288 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:12.853616 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:12.853620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:12.854212 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:12.854411 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:12.854416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:12.855181 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:12.855182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:12.855985 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:12.855987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:12.856372 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:12.856373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:12.856473 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:12.856937 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:12.856945 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:12.858397 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:12.859884 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:12.859892 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:12.859894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:12.860131 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:12.860384 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:12.860385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:12.860386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:12.860498 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:12.860819 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:12.860827 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:12.861006 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 08:45:12.861114 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:45:12.861275 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":124,"threads_started":1}
I20260809 08:45:12.862115 19966 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37975
I20260809 08:45:12.862221 19966 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:12.862421 19966 heartbeater.cc:511] Master 127.31.63.126:37975 requested a full tablet report, sending...
I20260809 08:45:12.863034 19674 ts_manager.cc:194] Registered new tserver with Master: 5f3bd4b00fce4eb9800f05248c6e3987 (127.31.63.65:38159)
I20260809 08:45:12.863926 19674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:44209
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:45:12.888443 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37975
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:12.973047 19973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:12.973285 19973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:12.973323 19973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:12.974506 19973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:12.974581 19973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:12.974623 19973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:45:12.976083 19973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:12.976349 19973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:12.976585 19973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:12.982034 19987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:12.982035 19983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:12.982059 19984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:12.982263 19973 server_base.cc:1034] running on GCE node
I20260809 08:45:12.982475 19973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:12.982728 19973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:12.983923 19973 hybrid_clock.cc:648] HybridClock initialized: now 1786265112983906 us; error 27 us; skew 500 ppm
Aug 09 08:45:12 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265112, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:12.991266 19973 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:45:12.991775 19973 webserver.cc:460] Webserver started at http://127.31.63.66:34103/ using document root <none> and password file <none>
I20260809 08:45:12.992027 19973 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:12.992115 19973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:12.992278 19973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:12.993472 19973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "46ae811764c94cf1ae024700ea2a8407"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "a447242ff26eafcbfd3640154488c15e"
server_key_iv: "a870aa2524e884ba5b2c309d041a342a"
server_key_version: "encryptionkey@0"
I20260809 08:45:12.993990 19973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "46ae811764c94cf1ae024700ea2a8407"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "a447242ff26eafcbfd3640154488c15e"
server_key_iv: "a870aa2524e884ba5b2c309d041a342a"
server_key_version: "encryptionkey@0"
I20260809 08:45:12.995472 19973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:45:12.996279 19997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:12.996488 19973 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:12.996600 19973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "46ae811764c94cf1ae024700ea2a8407"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "a447242ff26eafcbfd3640154488c15e"
server_key_iv: "a870aa2524e884ba5b2c309d041a342a"
server_key_version: "encryptionkey@0"
I20260809 08:45:12.996688 19973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:13.012993 19973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:13.015089 19973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:13.015267 19973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:13.015522 19973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:13.015862 19973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:13.015923 19973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.015964 19973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:13.015993 19973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.022553 19973 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:39919
I20260809 08:45:13.022629 20168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:39919 every 8 connection(s)
I20260809 08:45:13.022938 19973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:45:13.024222 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 19973
I20260809 08:45:13.024302 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:45:13.024490 31997 external_mini_cluster.cc:1428] Setting key 8e6d0e05d84485e1d71c6a3f6ea2eb74
Aug 09 08:45:13 dist-test-slave-gkw7 krb5kdc[19614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265112, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:13.033644 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.023696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:35343 (local address 127.31.63.126:37975)
0809 08:45:13.023771 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.023774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.023986 (+ 212us) server_negotiation.cc:407] Connection header received
0809 08:45:13.024663 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.024664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.024680 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.024748 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:13.025872 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.026619 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.027818 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.028029 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.028119 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.030590 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:13.030592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:13.030592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:13.030600 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:13.032140 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.032607 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.032609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.032609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.032671 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.032946 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.032947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.032948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.033068 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:13.033150 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.033436 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.033525 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:45:13.033855 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.023680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:35343)
0809 08:45:13.023855 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.023893 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.024504 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.024823 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.024827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.025522 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:13.025779 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.025784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.026754 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.026755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.027639 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.027642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.028133 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.028135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.028320 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.028986 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:13.028995 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:13.030479 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:13.032268 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:13.032276 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:13.032278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:13.032513 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:13.032754 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:13.032755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:13.032755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:13.032871 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:13.033140 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:13.033148 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:13.033294 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.033402 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.033589 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260809 08:45:13.034355 20169 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37975
I20260809 08:45:13.034443 20169 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:13.034641 20169 heartbeater.cc:511] Master 127.31.63.126:37975 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
I20260809 08:45:13.035086 19674 ts_manager.cc:194] Registered new tserver with Master: 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919)
I20260809 08:45:13.035470 19674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:35343
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:45:13.053076 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37975
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:13.139271 20176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:13.139467 20176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:13.139490 20176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:13.140544 20176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:13.140584 20176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:13.140610 20176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:45:13.142171 20176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:13.142354 20176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:13.142524 20176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:13.147501 20182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:13.147500 20184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:13.147571 20176 server_base.cc:1034] running on GCE node
W20260809 08:45:13.147513 20181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:13.147934 20176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:13.148211 20176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:13.149374 20176 hybrid_clock.cc:648] HybridClock initialized: now 1786265113149346 us; error 45 us; skew 500 ppm
Aug 09 08:45:13 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265113, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:13.151816 20176 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:45:13.152243 20176 webserver.cc:460] Webserver started at http://127.31.63.67:39771/ using document root <none> and password file <none>
I20260809 08:45:13.152498 20176 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:13.152568 20176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:13.152727 20176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:13.153838 20176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "978602d304e8405dad341e2335242eab"
format_stamp: "Formatted at 2026-08-09 08:45:13 on dist-test-slave-gkw7"
server_key: "7b031377afd986b52227180e0cec118f"
server_key_iv: "fb1a1306ee04767de2ad1ed4158eeca6"
server_key_version: "encryptionkey@0"
I20260809 08:45:13.154202 20176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "978602d304e8405dad341e2335242eab"
format_stamp: "Formatted at 2026-08-09 08:45:13 on dist-test-slave-gkw7"
server_key: "7b031377afd986b52227180e0cec118f"
server_key_iv: "fb1a1306ee04767de2ad1ed4158eeca6"
server_key_version: "encryptionkey@0"
I20260809 08:45:13.155571 20176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:45:13.156385 20195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.156570 20176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:13.156664 20176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "978602d304e8405dad341e2335242eab"
format_stamp: "Formatted at 2026-08-09 08:45:13 on dist-test-slave-gkw7"
server_key: "7b031377afd986b52227180e0cec118f"
server_key_iv: "fb1a1306ee04767de2ad1ed4158eeca6"
server_key_version: "encryptionkey@0"
I20260809 08:45:13.156757 20176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:13.177459 20176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:13.179888 20176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:13.180065 20176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:13.180330 20176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:13.180670 20176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:13.180737 20176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.180781 20176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:13.180830 20176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.187148 20176 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:38847
I20260809 08:45:13.187284 20369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:38847 every 8 connection(s)
I20260809 08:45:13.187613 20176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:45:13.189239 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 20176
I20260809 08:45:13.189323 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:45:13.189500 31997 external_mini_cluster.cc:1428] Setting key 5129395d85f3ac9f080d322426c63ba5
Aug 09 08:45:13 dist-test-slave-gkw7 krb5kdc[19614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265113, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:13.199867 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.188686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:55073 (local address 127.31.63.126:37975)
0809 08:45:13.188744 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.188746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.189053 (+ 307us) server_negotiation.cc:407] Connection header received
0809 08:45:13.190252 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.190254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.190267 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.190359 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:13.191744 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.192301 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.193360 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.193597 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.193661 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.196018 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:13.196020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:13.196021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:13.196030 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:13.197963 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.198576 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.198598 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.198599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.198673 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.199017 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.199018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.199019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.199186 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:13.199289 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.199674 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.199759 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:45:13.200096 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.188662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:55073)
0809 08:45:13.188889 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.188963 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.190115 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.190467 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.190472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.191363 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:13.191667 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.191673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.192405 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.192406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.193279 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.193281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.193679 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.193681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.193786 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.194223 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:13.194232 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:13.195921 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:13.198050 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:13.198064 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:13.198066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:13.198496 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:13.198735 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:13.198736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:13.198737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:13.198894 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:13.199292 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:13.199301 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:13.199513 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.199644 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.199833 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":133,"threads_started":1}
I20260809 08:45:13.200690 20371 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37975
I20260809 08:45:13.200802 20371 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:13.201007 20371 heartbeater.cc:511] Master 127.31.63.126:37975 requested a full tablet report, sending...
I20260809 08:45:13.201496 19674 ts_manager.cc:194] Registered new tserver with Master: 978602d304e8405dad341e2335242eab (127.31.63.67:38847)
I20260809 08:45:13.202019 19674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:55073
I20260809 08:45:13.202788 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:45:13.211082 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.204669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.31.63.126:37975)
0809 08:45:13.204732 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.204734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.204811 (+ 77us) server_negotiation.cc:407] Connection header received
0809 08:45:13.204898 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.204899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.204911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.204988 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:13.205787 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.206586 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.207335 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.207546 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.207618 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.208748 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:13.208750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:13.208750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:13.208756 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:13.210125 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.210467 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.210468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.210468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.210520 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.210668 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.210669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.210669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.210801 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:13.210873 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.210964 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.211021 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:45:13.212850 19674 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:45:13.214614 19674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:45:13.221513 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.216991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:39919 (local address 127.0.0.1:49834)
0809 08:45:13.217645 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.217657 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.217719 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.217999 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.218000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.218002 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:13.218041 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.218042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.218944 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.218945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.220217 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.220220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.221200 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.221201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.221291 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.221307 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.221380 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.221436 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":84,"threads_started":1}
I20260809 08:45:13.221510 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.217262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49834 (local address 127.31.63.66:39919)
0809 08:45:13.217469 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.217473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.217786 (+ 313us) server_negotiation.cc:407] Connection header received
0809 08:45:13.217830 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.217832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.217848 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.217916 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:13.218144 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.218838 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.220314 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.221096 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.221176 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.221240 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.221404 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.221441 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260809 08:45:13.221987 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.216982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:38847 (local address 127.0.0.1:33266)
0809 08:45:13.217171 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.217224 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.217296 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.217850 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.217851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.217853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:13.217916 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.217918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.219183 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.219184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.220223 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.220224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.221755 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.221756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.221831 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.221835 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.221885 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.221924 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":8,"thread_start_us":69,"threads_started":1}
I20260809 08:45:13.222352 20074 tablet_service.cc:1467] Processing CreateTablet for tablet 5b4509e2d2494dbab8c4dda6a6ba414c (DEFAULT_TABLE table=test-table [id=f796cab17e064a08bb0f41aa6602a81e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:13.222656 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.216987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:38159 (local address 127.0.0.1:52152)
0809 08:45:13.217342 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.217351 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.217398 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.217746 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.217750 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.217752 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:13.217821 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.217823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.220325 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.220325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.221129 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.221131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.222384 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.222385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.222461 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.222465 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.222523 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.222570 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":110,"threads_started":1}
I20260809 08:45:13.222704 20388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.217088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.31.63.65:38159)
0809 08:45:13.217290 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.217295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.217456 (+ 161us) server_negotiation.cc:407] Connection header received
0809 08:45:13.217495 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.217496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.217509 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.217625 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:13.217911 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.219071 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.221249 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.222280 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.222381 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.222439 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.222566 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.222610 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":80,"threads_started":1}
I20260809 08:45:13.222728 20074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b4509e2d2494dbab8c4dda6a6ba414c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:13.223074 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.217260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.31.63.67:38847)
0809 08:45:13.217608 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.217612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.217624 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:45:13.217669 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.217672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.217689 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.217766 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:13.218006 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.219085 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.220314 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.221099 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.221176 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.221244 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.222972 (+ 1728us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.223014 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":97,"threads_started":1}
I20260809 08:45:13.223757 20262 tablet_service.cc:1467] Processing CreateTablet for tablet 5b4509e2d2494dbab8c4dda6a6ba414c (DEFAULT_TABLE table=test-table [id=f796cab17e064a08bb0f41aa6602a81e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:13.223943 19867 tablet_service.cc:1467] Processing CreateTablet for tablet 5b4509e2d2494dbab8c4dda6a6ba414c (DEFAULT_TABLE table=test-table [id=f796cab17e064a08bb0f41aa6602a81e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:13.224084 20262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b4509e2d2494dbab8c4dda6a6ba414c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:13.224306 19867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b4509e2d2494dbab8c4dda6a6ba414c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:13.226421 20397 tablet_bootstrap.cc:492] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: Bootstrap starting.
I20260809 08:45:13.227028 20398 tablet_bootstrap.cc:492] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: Bootstrap starting.
I20260809 08:45:13.227217 20397 tablet_bootstrap.cc:654] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:13.227804 20398 tablet_bootstrap.cc:654] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:13.227874 20397 log.cc:824] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:13.228302 20399 tablet_bootstrap.cc:492] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Bootstrap starting.
I20260809 08:45:13.228423 20398 log.cc:824] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:13.228721 20397 tablet_bootstrap.cc:492] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: No bootstrap required, opened a new log
I20260809 08:45:13.228801 20397 ts_tablet_manager.cc:1397] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:45:13.229169 20398 tablet_bootstrap.cc:492] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: No bootstrap required, opened a new log
I20260809 08:45:13.229264 20398 ts_tablet_manager.cc:1397] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:45:13.230377 20399 tablet_bootstrap.cc:654] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:13.230672 20397 raft_consensus.cc:348] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.230845 20397 raft_consensus.cc:374] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:13.230887 20397 raft_consensus.cc:729] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ae811764c94cf1ae024700ea2a8407, State: Initialized, Role: FOLLOWER
I20260809 08:45:13.230998 20397 consensus_queue.cc:260] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.231016 20399 log.cc:824] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:13.231338 20397 ts_tablet_manager.cc:1428] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:45:13.231494 20169 heartbeater.cc:503] Master 127.31.63.126:37975 was elected leader, sending a full tablet report...
I20260809 08:45:13.232066 20399 tablet_bootstrap.cc:492] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: No bootstrap required, opened a new log
I20260809 08:45:13.232010 20398 raft_consensus.cc:348] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.232173 20398 raft_consensus.cc:374] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:13.232201 20399 ts_tablet_manager.cc:1397] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 08:45:13.232234 20398 raft_consensus.cc:729] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978602d304e8405dad341e2335242eab, State: Initialized, Role: FOLLOWER
I20260809 08:45:13.232365 20398 consensus_queue.cc:260] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.232645 20398 ts_tablet_manager.cc:1428] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:45:13.232762 20371 heartbeater.cc:503] Master 127.31.63.126:37975 was elected leader, sending a full tablet report...
I20260809 08:45:13.234300 20399 raft_consensus.cc:348] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.234423 20399 raft_consensus.cc:374] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:13.234444 20399 raft_consensus.cc:729] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3bd4b00fce4eb9800f05248c6e3987, State: Initialized, Role: FOLLOWER
I20260809 08:45:13.234551 20399 consensus_queue.cc:260] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.234820 20399 ts_tablet_manager.cc:1428] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:45:13.234853 19966 heartbeater.cc:503] Master 127.31.63.126:37975 was elected leader, sending a full tablet report...
I20260809 08:45:13.272033 20402 raft_consensus.cc:482] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:13.272161 20402 raft_consensus.cc:504] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.272599 20402 leader_election.cc:288] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 978602d304e8405dad341e2335242eab (127.31.63.67:38847), 5f3bd4b00fce4eb9800f05248c6e3987 (127.31.63.65:38159)
W20260809 08:45:13.273581 20170 tablet.cc:2369] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:45:13.277251 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.272788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:38847 (local address 127.31.63.66:34223)
0809 08:45:13.272981 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.272995 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.273065 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.273506 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.273507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.273509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:13.273576 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.273578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.274471 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.274473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.275967 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.275970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.276982 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.276983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.277051 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.277055 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.277112 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.277165 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":48}
I20260809 08:45:13.277261 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.272889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:34223 (local address 127.31.63.67:38847)
0809 08:45:13.272968 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.272970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.273064 (+ 94us) server_negotiation.cc:407] Connection header received
0809 08:45:13.273163 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.273165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.273176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.273468 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:13.273653 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.274376 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.276092 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.276871 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.276945 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.276954 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.277142 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.277192 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:45:13.277381 20388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.273205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:42697 (local address 127.31.63.65:38159)
0809 08:45:13.273257 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.273259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.273269 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:45:13.273308 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.273309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.273315 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.273391 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:13.273616 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.274388 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.276133 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.276846 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.276924 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.276934 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.277190 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.277277 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:45:13.277633 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.272837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:38159 (local address 127.31.63.66:42697)
0809 08:45:13.273060 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.273072 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.273130 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.273410 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.273412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.273414 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:13.273498 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.273501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.274486 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.274487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.276006 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.276008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.276962 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.276963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.277071 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.277074 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.277486 (+ 412us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.277555 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":64,"thread_start_us":119,"threads_started":1}
I20260809 08:45:13.277824 20298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b4509e2d2494dbab8c4dda6a6ba414c" candidate_uuid: "46ae811764c94cf1ae024700ea2a8407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978602d304e8405dad341e2335242eab" is_pre_election: true
I20260809 08:45:13.278008 20298 raft_consensus.cc:2454] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46ae811764c94cf1ae024700ea2a8407 in term 0.
I20260809 08:45:13.278259 19998 leader_election.cc:302] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46ae811764c94cf1ae024700ea2a8407, 978602d304e8405dad341e2335242eab; no voters:
I20260809 08:45:13.278466 20402 raft_consensus.cc:2781] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:45:13.278548 20402 raft_consensus.cc:482] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:45:13.278580 20402 raft_consensus.cc:3037] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:13.278644 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b4509e2d2494dbab8c4dda6a6ba414c" candidate_uuid: "46ae811764c94cf1ae024700ea2a8407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" is_pre_election: true
I20260809 08:45:13.278825 19895 raft_consensus.cc:2454] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46ae811764c94cf1ae024700ea2a8407 in term 0.
I20260809 08:45:13.279618 20402 raft_consensus.cc:504] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.279765 20402 leader_election.cc:288] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [CANDIDATE]: Term 1 election: Requested vote from peers 978602d304e8405dad341e2335242eab (127.31.63.67:38847), 5f3bd4b00fce4eb9800f05248c6e3987 (127.31.63.65:38159)
I20260809 08:45:13.279919 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b4509e2d2494dbab8c4dda6a6ba414c" candidate_uuid: "46ae811764c94cf1ae024700ea2a8407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3bd4b00fce4eb9800f05248c6e3987"
I20260809 08:45:13.279928 20298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b4509e2d2494dbab8c4dda6a6ba414c" candidate_uuid: "46ae811764c94cf1ae024700ea2a8407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "978602d304e8405dad341e2335242eab"
I20260809 08:45:13.280009 19895 raft_consensus.cc:3037] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:13.280011 20298 raft_consensus.cc:3037] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:13.281044 19895 raft_consensus.cc:2454] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46ae811764c94cf1ae024700ea2a8407 in term 1.
I20260809 08:45:13.281064 20298 raft_consensus.cc:2454] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46ae811764c94cf1ae024700ea2a8407 in term 1.
I20260809 08:45:13.281214 20000 leader_election.cc:302] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46ae811764c94cf1ae024700ea2a8407, 5f3bd4b00fce4eb9800f05248c6e3987; no voters:
I20260809 08:45:13.281319 20402 raft_consensus.cc:2781] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:45:13.281488 20402 raft_consensus.cc:686] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 1 LEADER]: Becoming Leader. State: Replica: 46ae811764c94cf1ae024700ea2a8407, State: Running, Role: LEADER
I20260809 08:45:13.281597 20402 consensus_queue.cc:237] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.282505 19674 catalog_manager.cc:5495] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "46ae811764c94cf1ae024700ea2a8407" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } health_report { overall_health: HEALTHY } } }
I20260809 08:45:13.298233 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.295643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49844 (local address 127.31.63.66:39919)
0809 08:45:13.295727 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.295729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.295741 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:45:13.295777 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.295777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.295797 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.295870 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:45:13.296032 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.296517 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.297302 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.297505 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.297658 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.297850 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:45:13.297963 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:45:13.298055 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.298085 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.298108 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:45:13.300415 19895 raft_consensus.cc:1261] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Refusing update from remote peer 46ae811764c94cf1ae024700ea2a8407: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:13.300493 20298 raft_consensus.cc:1261] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab [term 1 FOLLOWER]: Refusing update from remote peer 46ae811764c94cf1ae024700ea2a8407: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:13.300813 20407 consensus_queue.cc:1035] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:13.300984 20402 consensus_queue.cc:1035] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:13.304178 20413 mvcc.cc:205] Tried to move back new op lower bound from 7316541904075087872 to 7316541904002023424. Current Snapshot: MvccSnapshot[applied={T|T < 7316541904075087872}]
I20260809 08:45:13.305969 19673 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:45:13.306077 19673 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:45:13.306429 20412 mvcc.cc:205] Tried to move back new op lower bound from 7316541904075087872 to 7316541904002023424. Current Snapshot: MvccSnapshot[applied={T|T < 7316541904075087872 or (T in {7316541904075087872})}]
I20260809 08:45:13.306900 19673 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Sending DeleteTablet for 3 replicas of tablet 5b4509e2d2494dbab8c4dda6a6ba414c
I20260809 08:45:13.307205 20074 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b4509e2d2494dbab8c4dda6a6ba414c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:13 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:49834
I20260809 08:45:13.307340 19867 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b4509e2d2494dbab8c4dda6a6ba414c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:13 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:52152
I20260809 08:45:13.307348 20262 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b4509e2d2494dbab8c4dda6a6ba414c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:13 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:33266
I20260809 08:45:13.307464 20425 tablet_replica.cc:330] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: stopping tablet replica
I20260809 08:45:13.307538 20259 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b4509e2d2494dbab8c4dda6a6ba414c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:13 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:33266
I20260809 08:45:13.307562 20425 raft_consensus.cc:2229] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:45:13.307658 20425 raft_consensus.cc:2258] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:13.307724 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 19773
I20260809 08:45:13.317243 20425 ts_tablet_manager.cc:1905] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:13.319247 20425 ts_tablet_manager.cc:1918] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:13.319329 20425 log.cc:1196] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/5b4509e2d2494dbab8c4dda6a6ba414c
I20260809 08:45:13.319579 20425 ts_tablet_manager.cc:1939] T 5b4509e2d2494dbab8c4dda6a6ba414c P 46ae811764c94cf1ae024700ea2a8407: Deleting consensus metadata
I20260809 08:45:13.320030 19655 catalog_manager.cc:4841] TS 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919): tablet 5b4509e2d2494dbab8c4dda6a6ba414c (table test-table [id=f796cab17e064a08bb0f41aa6602a81e]) successfully deleted
I20260809 08:45:13.320271 20426 tablet_replica.cc:330] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: stopping tablet replica
I20260809 08:45:13.320387 20426 raft_consensus.cc:2229] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:45:13.320493 20426 raft_consensus.cc:2258] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:13.321905 20426 ts_tablet_manager.cc:1905] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:13.323194 20426 ts_tablet_manager.cc:1918] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:13.323269 20426 log.cc:1196] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/5b4509e2d2494dbab8c4dda6a6ba414c
I20260809 08:45:13.323485 20426 ts_tablet_manager.cc:1939] T 5b4509e2d2494dbab8c4dda6a6ba414c P 978602d304e8405dad341e2335242eab: Deleting consensus metadata
I20260809 08:45:13.323936 19654 catalog_manager.cc:4841] TS 978602d304e8405dad341e2335242eab (127.31.63.67:38847): tablet 5b4509e2d2494dbab8c4dda6a6ba414c (table test-table [id=f796cab17e064a08bb0f41aa6602a81e]) successfully deleted
W20260809 08:45:13.324092 19654 catalog_manager.cc:4808] TS 978602d304e8405dad341e2335242eab (127.31.63.67:38847): delete failed for tablet 5b4509e2d2494dbab8c4dda6a6ba414c because the tablet was not found. No further retry: Not found: Tablet not found: 5b4509e2d2494dbab8c4dda6a6ba414c
W20260809 08:45:13.326644 19657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:38159 (error 108)
I20260809 08:45:13.326872 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 19973
I20260809 08:45:13.330502 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.329386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:38159 (local address 127.0.0.1:52164)
0809 08:45:13.330396 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.330454 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:38159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":977}
W20260809 08:45:13.330668 19657 catalog_manager.cc:4584] TS 5f3bd4b00fce4eb9800f05248c6e3987 (127.31.63.65:38159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b4509e2d2494dbab8c4dda6a6ba414c: Network error: Client connection negotiation failed: client connection to 127.31.63.65:38159: connect: Connection refused (error 111)
I20260809 08:45:13.337358 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 20176
I20260809 08:45:13.346189 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 19630
I20260809 08:45:13.358613 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37975
--webserver_interface=127.31.63.126
--webserver_port=41029
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:37975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:13.442559 20431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:13.442765 20431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:13.442788 20431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:13.443990 20431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:13.444036 20431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:13.444049 20431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:13.444096 20431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:13.444126 20431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:13.445644 20431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:37975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=41029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:13.445915 20431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:13.446178 20431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:13.456153 20447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:13.456183 20444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:13.456213 20443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:13.456385 20431 server_base.cc:1034] running on GCE node
I20260809 08:45:13.456730 20431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:13.457113 20431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:13.458279 20431 hybrid_clock.cc:648] HybridClock initialized: now 1786265113458242 us; error 56 us; skew 500 ppm
Aug 09 08:45:13 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265113, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:13.461057 20431 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:45:13.461570 20431 webserver.cc:460] Webserver started at http://127.31.63.126:41029/ using document root <none> and password file <none>
I20260809 08:45:13.461870 20431 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:13.461958 20431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:13.463428 20431 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:45:13.464110 20463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.464298 20431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:45:13.464409 20431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "fcbad767f2244341a799a96daa358c1e"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "21459d89286fa2943a6c94b828579538"
server_key_iv: "e5ff227e3756564b194080d25e67bad6"
server_key_version: "encryptionkey@0"
I20260809 08:45:13.464816 20431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:13.476495 20431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:13.478925 20431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:13.479101 20431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:13.483475 20431 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:37975
I20260809 08:45:13.483533 20547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:37975 every 8 connection(s)
I20260809 08:45:13.483951 20431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:45:13.484433 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 20431
I20260809 08:45:13.484846 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:38159
--local_ip_for_outbound_sockets=127.31.63.65
--tserver_master_addrs=127.31.63.126:37975
--webserver_port=39155
--webserver_interface=127.31.63.65
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:13.486650 20549 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:45:13.487421 20549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Bootstrap starting.
I20260809 08:45:13.499033 20549 log.cc:824] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:13.502182 20549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:45:13.502502 20549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Bootstrap complete.
I20260809 08:45:13.505172 20549 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:13.505441 20549 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcbad767f2244341a799a96daa358c1e, State: Initialized, Role: FOLLOWER
I20260809 08:45:13.505678 20549 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:13.505782 20549 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:45:13.505846 20549 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:45:13.505898 20549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:45:13.506919 20549 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:13.507051 20549 leader_election.cc:302] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcbad767f2244341a799a96daa358c1e; no voters:
I20260809 08:45:13.507254 20549 leader_election.cc:288] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:45:13.507318 20558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:45:13.507472 20558 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 2 LEADER]: Becoming Leader. State: Replica: fcbad767f2244341a799a96daa358c1e, State: Running, Role: LEADER
I20260809 08:45:13.507535 20549 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:13.507586 20558 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:13.507861 20558 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcbad767f2244341a799a96daa358c1e. Latest consensus state: current_term: 2 leader_uuid: "fcbad767f2244341a799a96daa358c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } } }
I20260809 08:45:13.507933 20558 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:13.507864 20559 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fcbad767f2244341a799a96daa358c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } } }
I20260809 08:45:13.508131 20559 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:13.509573 20582 catalog_manager.cc:1259] Loaded cluster ID: fa108a38c71948c2b5b6e37a394b4197
I20260809 08:45:13.509665 20582 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: loading cluster ID for follower catalog manager: success
W20260809 08:45:13.509948 20582 catalog_manager.cc:1576] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260809 08:45:13.510071 20582 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260809 08:45:13.510191 20565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:45:13.510710 20565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:45:13.510800 20565 catalog_manager.cc:1259] Loaded cluster ID: fa108a38c71948c2b5b6e37a394b4197
I20260809 08:45:13.510830 20565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:45:13.510991 20565 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 08:45:13.515434 20565 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 08:45:13.517907 20565 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:45:13.518123 20565 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Loaded TSK: 0
I20260809 08:45:13.518338 20565 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:45:13.579752 20554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:13.579977 20554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:13.580001 20554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:13.581173 20554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:13.581214 20554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:13.581241 20554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:45:13.582790 20554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:38159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=39155
--tserver_master_addrs=127.31.63.126:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:13.583027 20554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:13.583297 20554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:13.588537 20593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:13.588682 20594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:13.588727 20554 server_base.cc:1034] running on GCE node
W20260809 08:45:13.588516 20598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:13.589075 20554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:13.589359 20554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:13.590549 20554 hybrid_clock.cc:648] HybridClock initialized: now 1786265113590525 us; error 42 us; skew 500 ppm
Aug 09 08:45:13 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265113, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:13.593397 20554 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:45:13.593935 20554 webserver.cc:460] Webserver started at http://127.31.63.65:39155/ using document root <none> and password file <none>
I20260809 08:45:13.594198 20554 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:13.594283 20554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:13.595666 20554 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:45:13.596401 20607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.596554 20554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:45:13.596665 20554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "5f3bd4b00fce4eb9800f05248c6e3987"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "7a9f28125836c6bf129722f2fc1496fd"
server_key_iv: "4c288b83df27453e0b2257d45681d8d7"
server_key_version: "encryptionkey@0"
I20260809 08:45:13.597021 20554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:13.608527 20554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:13.610936 20554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:13.611093 20554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:13.611347 20554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:13.611876 20614 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:45:13.613368 20554 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:45:13.613422 20554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:45:13.613458 20554 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:45:13.614181 20554 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:45:13.614229 20554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 08:45:13.614277 20614 tablet_bootstrap.cc:492] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Bootstrap starting.
I20260809 08:45:13.622510 20554 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:38159
I20260809 08:45:13.622633 20790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:38159 every 8 connection(s)
I20260809 08:45:13.623162 20554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:45:13.623862 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 20554
I20260809 08:45:13.624161 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:39919
--local_ip_for_outbound_sockets=127.31.63.66
--tserver_master_addrs=127.31.63.126:37975
--webserver_port=34103
--webserver_interface=127.31.63.66
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:13.627878 20614 log.cc:824] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:13.631014 20614 tablet_bootstrap.cc:492] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:45:13.631337 20614 tablet_bootstrap.cc:492] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Bootstrap complete.
I20260809 08:45:13.633100 20614 ts_tablet_manager.cc:1397] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.014s
Aug 09 08:45:13 dist-test-slave-gkw7 krb5kdc[19614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265113, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:13.634887 20614 raft_consensus.cc:348] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.635990 20614 raft_consensus.cc:729] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3bd4b00fce4eb9800f05248c6e3987, State: Initialized, Role: FOLLOWER
I20260809 08:45:13.636168 20614 consensus_queue.cc:260] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } }
I20260809 08:45:13.636503 20614 ts_tablet_manager.cc:1428] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:45:13.639068 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.624035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:34087 (local address 127.31.63.126:37975)
0809 08:45:13.624527 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.624538 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.626754 (+ 2216us) server_negotiation.cc:407] Connection header received
0809 08:45:13.626929 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.626937 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.627018 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.627202 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:13.628056 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.629489 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.631040 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.631630 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.631734 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.634640 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:13.634644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:13.634644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:13.634656 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:13.636807 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.637374 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.637377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.637377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.637512 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.637932 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.637934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.637934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.638236 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:13.638310 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.638707 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.638867 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":109,"threads_started":1}
I20260809 08:45:13.639562 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.623925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:34087)
0809 08:45:13.625687 (+ 1762us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.625748 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.626550 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.627262 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.627267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.627633 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:13.627905 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.627912 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.629670 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.629671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.630931 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.630935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.631891 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.631892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.632022 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.632668 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:13.632679 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:13.634516 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:13.636920 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:13.636928 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:13.636930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:13.637251 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:13.637662 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:13.637663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:13.637664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:13.637840 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:13.638351 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:13.638357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:13.638516 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.639184 (+ 668us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.639369 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1703,"thread_start_us":116,"threads_started":1}
I20260809 08:45:13.640355 20791 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37975
I20260809 08:45:13.640455 20791 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:13.640636 20791 heartbeater.cc:511] Master 127.31.63.126:37975 requested a full tablet report, sending...
I20260809 08:45:13.641469 20497 ts_manager.cc:194] Registered new tserver with Master: 5f3bd4b00fce4eb9800f05248c6e3987 (127.31.63.65:38159)
I20260809 08:45:13.642824 20497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:34087
I20260809 08:45:13.643875 20791 heartbeater.cc:503] Master 127.31.63.126:37975 was elected leader, sending a full tablet report...
I20260809 08:45:13.647758 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.643112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52174 (local address 127.31.63.65:38159)
0809 08:45:13.643271 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.643276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.643604 (+ 328us) server_negotiation.cc:407] Connection header received
0809 08:45:13.643641 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.643643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.643715 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.643784 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:13.644095 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.645286 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.646617 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.647331 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.647410 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.647464 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.647645 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.647689 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260809 08:45:13.647758 20798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.642222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:38159 (local address 127.0.0.1:52174)
0809 08:45:13.643401 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.643440 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.643512 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.643860 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.643863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.643865 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:13.643929 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.643931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.645384 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.645385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.646521 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.646524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.647429 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.647430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.647523 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.647537 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.647615 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.647667 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1146,"thread_start_us":96,"threads_started":1}
I20260809 08:45:13.648550 20683 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b4509e2d2494dbab8c4dda6a6ba414c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:13 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:52174
I20260809 08:45:13.649072 20801 tablet_replica.cc:330] stopping tablet replica
I20260809 08:45:13.649185 20801 raft_consensus.cc:2229] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:45:13.649253 20801 raft_consensus.cc:2258] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:13.649659 20801 ts_tablet_manager.cc:1905] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:13.651928 20801 ts_tablet_manager.cc:1918] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:13.652010 20801 log.cc:1196] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/5b4509e2d2494dbab8c4dda6a6ba414c
I20260809 08:45:13.652235 20801 ts_tablet_manager.cc:1939] T 5b4509e2d2494dbab8c4dda6a6ba414c P 5f3bd4b00fce4eb9800f05248c6e3987: Deleting consensus metadata
I20260809 08:45:13.652776 20466 catalog_manager.cc:4841] TS 5f3bd4b00fce4eb9800f05248c6e3987 (127.31.63.65:38159): tablet 5b4509e2d2494dbab8c4dda6a6ba414c (table test-table [id=f796cab17e064a08bb0f41aa6602a81e]) successfully deleted
W20260809 08:45:13.721436 20795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:13.721681 20795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:13.721705 20795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:13.722965 20795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:13.723013 20795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:13.723040 20795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:45:13.724438 20795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:39919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=34103
--tserver_master_addrs=127.31.63.126:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:13.724694 20795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:13.724944 20795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:13.730237 20809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:13.730271 20812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:13.730281 20808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:13.730451 20795 server_base.cc:1034] running on GCE node
I20260809 08:45:13.730679 20795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:13.730883 20795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:13.732024 20795 hybrid_clock.cc:648] HybridClock initialized: now 1786265113731997 us; error 45 us; skew 500 ppm
Aug 09 08:45:13 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265113, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:13.734668 20795 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:45:13.735102 20795 webserver.cc:460] Webserver started at http://127.31.63.66:34103/ using document root <none> and password file <none>
I20260809 08:45:13.735337 20795 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:13.735396 20795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:13.736781 20795 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:45:13.737377 20821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.737593 20795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:45:13.737696 20795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "46ae811764c94cf1ae024700ea2a8407"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "a447242ff26eafcbfd3640154488c15e"
server_key_iv: "a870aa2524e884ba5b2c309d041a342a"
server_key_version: "encryptionkey@0"
I20260809 08:45:13.738060 20795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:13.746999 20795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:13.749069 20795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:13.749240 20795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:13.749480 20795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:13.749850 20795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:13.749910 20795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.749955 20795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:13.749984 20795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.756290 20795 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:39919
I20260809 08:45:13.756349 20993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:39919 every 8 connection(s)
I20260809 08:45:13.756738 20795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:45:13 dist-test-slave-gkw7 krb5kdc[19614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265113, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:13.765297 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 20795
I20260809 08:45:13.765674 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:38847
--local_ip_for_outbound_sockets=127.31.63.67
--tserver_master_addrs=127.31.63.126:37975
--webserver_port=39771
--webserver_interface=127.31.63.67
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:13.766422 20996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.757513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:42071)
0809 08:45:13.757706 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.757750 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.758271 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.758500 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.758503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.759075 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:13.759288 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.759293 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.759997 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.759998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.760809 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.760811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.761177 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.761178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.761286 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.761911 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:13.761919 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:13.763303 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:13.764940 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:13.764947 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:13.764949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:13.765163 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:13.765421 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:13.765422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:13.765422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:13.765600 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:13.765923 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:13.765929 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:13.766048 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.766128 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.766260 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":117,"threads_started":1}
I20260809 08:45:13.766438 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.757573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:42071 (local address 127.31.63.126:37975)
0809 08:45:13.757642 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.757644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.757866 (+ 222us) server_negotiation.cc:407] Connection header received
0809 08:45:13.758363 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.758364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.758379 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.758440 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:13.759392 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.759907 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.760889 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.761093 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.761154 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.763388 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:13.763390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:13.763390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:13.763397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:13.764853 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.765257 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.765259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.765260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.765331 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.765684 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.765685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.765686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.765848 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:13.765925 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.766191 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.766292 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:45:13.767297 20994 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37975
I20260809 08:45:13.767413 20994 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:13.767597 20994 heartbeater.cc:511] Master 127.31.63.126:37975 requested a full tablet report, sending...
I20260809 08:45:13.768061 20497 ts_manager.cc:194] Registered new tserver with Master: 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919)
I20260809 08:45:13.768466 20497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:42071
W20260809 08:45:13.851365 20997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:13.851572 20997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:13.851595 20997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:13.852663 20997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:13.852696 20997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:13.852722 20997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:45:13.854198 20997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:38847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=39771
--tserver_master_addrs=127.31.63.126:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:13.854382 20997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:13.854557 20997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:13.859771 21008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:13.859810 21013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:13.859951 20997 server_base.cc:1034] running on GCE node
W20260809 08:45:13.859781 21007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:13.860275 20997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:13.860522 20997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:13.861660 20997 hybrid_clock.cc:648] HybridClock initialized: now 1786265113861639 us; error 39 us; skew 500 ppm
Aug 09 08:45:13 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265113, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:13.864136 20997 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:45:13.864565 20997 webserver.cc:460] Webserver started at http://127.31.63.67:39771/ using document root <none> and password file <none>
I20260809 08:45:13.864809 20997 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:13.864895 20997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:13.866303 20997 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:45:13.866884 21023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.867048 20997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:45:13.867156 20997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "978602d304e8405dad341e2335242eab"
format_stamp: "Formatted at 2026-08-09 08:45:13 on dist-test-slave-gkw7"
server_key: "7b031377afd986b52227180e0cec118f"
server_key_iv: "fb1a1306ee04767de2ad1ed4158eeca6"
server_key_version: "encryptionkey@0"
I20260809 08:45:13.867553 20997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:13.894129 20997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:13.896154 20997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:13.896335 20997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:13.896602 20997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:13.896941 20997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:13.896999 20997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.897042 20997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:13.897068 20997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.903888 20997 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:38847
I20260809 08:45:13.903972 21193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:38847 every 8 connection(s)
I20260809 08:45:13.904327 20997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:45:13 dist-test-slave-gkw7 krb5kdc[19614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265113, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:13.912345 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 20997
I20260809 08:45:13.914875 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.905133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:59025 (local address 127.31.63.126:37975)
0809 08:45:13.905204 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.905206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.905436 (+ 230us) server_negotiation.cc:407] Connection header received
0809 08:45:13.906065 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.906066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.906080 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.906129 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:13.907293 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.907930 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.909021 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.909229 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.909309 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.911527 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:13.911529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:13.911529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:13.911537 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:13.913427 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.913949 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.913950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.913950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.914019 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.914276 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.914277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.914278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.914409 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:13.914472 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.914732 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.914808 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:45:13.915093 21200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.905128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:59025)
0809 08:45:13.905312 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.905349 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.905950 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.906222 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.906228 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.906981 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:13.907182 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.907187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.908033 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.908034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.908916 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.908919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.909320 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.909321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.909448 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.910037 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:13.910045 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:13.911412 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:13.913504 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:13.913512 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:13.913515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:13.913881 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:13.914091 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:13.914092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:13.914092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:13.914196 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:13.914472 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:13.914477 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:13.914591 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.914668 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.914861 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260809 08:45:13.915817 21196 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37975
I20260809 08:45:13.915931 21196 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:13.916134 21196 heartbeater.cc:511] Master 127.31.63.126:37975 requested a full tablet report, sending...
I20260809 08:45:13.916599 20497 ts_manager.cc:194] Registered new tserver with Master: 978602d304e8405dad341e2335242eab (127.31.63.67:38847)
I20260809 08:45:13.916981 20497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:59025
I20260809 08:45:13.919924 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.912792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.31.63.126:37975)
0809 08:45:13.913089 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.913092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.913099 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:45:13.913120 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.913121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.913133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.913202 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:13.913937 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.914640 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.915600 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.915858 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.915927 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.917276 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:13.917277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:13.917278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:13.917286 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:13.918739 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.919108 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.919109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.919109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.919169 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:13.919400 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:13.919401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:13.919401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:13.919512 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:13.919695 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.919733 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.919815 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":55,"threads_started":1}
I20260809 08:45:13.920823 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:45:13.923790 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.921160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.31.63.126:37975)
0809 08:45:13.921223 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.921225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.921292 (+ 67us) server_negotiation.cc:407] Connection header received
0809 08:45:13.921355 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.921356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.921386 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.921456 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:45:13.921594 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.922123 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.922910 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.923147 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.923208 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.923383 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:45:13.923539 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:45:13.923629 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.923717 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.923740 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:45:13.924125 20497 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:45:13.925855 20497 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:45:13.928053 20683 tablet_service.cc:1467] Processing CreateTablet for tablet d26084767a57417e869414dd32522bfe (DEFAULT_TABLE table=test-table [id=3d863463b6ec43aea4bbd1eb91b2780e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:13.928285 20683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d26084767a57417e869414dd32522bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:13.930018 20614 tablet_bootstrap.cc:492] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: Bootstrap starting.
I20260809 08:45:13.930384 20614 tablet_bootstrap.cc:654] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:13.931103 20614 tablet_bootstrap.cc:492] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: No bootstrap required, opened a new log
I20260809 08:45:13.931177 20614 ts_tablet_manager.cc:1397] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:45:13.931293 20614 raft_consensus.cc:348] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:13.931396 20614 raft_consensus.cc:374] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:13.931442 20614 raft_consensus.cc:729] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3bd4b00fce4eb9800f05248c6e3987, State: Initialized, Role: FOLLOWER
I20260809 08:45:13.931500 20614 consensus_queue.cc:260] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:13.931593 20614 ts_tablet_manager.cc:1428] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:13.931648 20798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.927765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:38847 (local address 127.0.0.1:33268)
0809 08:45:13.927819 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.927834 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.927875 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.928305 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.928307 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.928308 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:13.928349 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.928351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.929428 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.929429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.930339 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.930343 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.931376 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.931377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.931445 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.931449 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.931513 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.931561 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 08:45:13.931689 21206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.927860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33268 (local address 127.31.63.67:38847)
0809 08:45:13.928060 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.928065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.928076 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:45:13.928155 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.928157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.928174 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.928302 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:13.928422 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.929320 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.930443 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.931269 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.931346 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.931411 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.931565 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.931608 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":111,"threads_started":1}
I20260809 08:45:13.932420 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.927774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:39919 (local address 127.0.0.1:49858)
0809 08:45:13.928322 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:13.928332 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:13.928381 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:13.928601 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:13.928602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:13.928605 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:13.928704 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.928706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.930021 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.930022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.931287 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:13.931290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.932168 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.932169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:13.932246 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.932250 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:13.932300 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:45:13.932339 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
I20260809 08:45:13.932564 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:13.928054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.31.63.66:39919)
0809 08:45:13.928264 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:13.928268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:13.928438 (+ 170us) server_negotiation.cc:407] Connection header received
0809 08:45:13.928485 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:13.928488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:13.928503 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:13.928580 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:13.928805 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.929897 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.931381 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:13.932059 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:13.932164 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:13.932220 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:13.932337 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 08:45:13.932488 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
I20260809 08:45:13.932767 21102 tablet_service.cc:1467] Processing CreateTablet for tablet d26084767a57417e869414dd32522bfe (DEFAULT_TABLE table=test-table [id=3d863463b6ec43aea4bbd1eb91b2780e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:13.933094 21102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d26084767a57417e869414dd32522bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:13.934160 20894 tablet_service.cc:1467] Processing CreateTablet for tablet d26084767a57417e869414dd32522bfe (DEFAULT_TABLE table=test-table [id=3d863463b6ec43aea4bbd1eb91b2780e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:13.934470 20894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d26084767a57417e869414dd32522bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:13.936451 21209 tablet_bootstrap.cc:492] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: Bootstrap starting.
I20260809 08:45:13.937040 21209 tablet_bootstrap.cc:654] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:13.937310 21210 tablet_bootstrap.cc:492] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: Bootstrap starting.
I20260809 08:45:13.937677 21209 log.cc:824] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:13.937973 21210 tablet_bootstrap.cc:654] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:13.938428 21209 tablet_bootstrap.cc:492] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: No bootstrap required, opened a new log
I20260809 08:45:13.938486 21209 ts_tablet_manager.cc:1397] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:45:13.938625 21210 log.cc:824] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:13.939452 21210 tablet_bootstrap.cc:492] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: No bootstrap required, opened a new log
I20260809 08:45:13.939548 21210 ts_tablet_manager.cc:1397] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:45:13.940186 21209 raft_consensus.cc:348] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:13.940315 21209 raft_consensus.cc:374] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:13.940336 21209 raft_consensus.cc:729] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978602d304e8405dad341e2335242eab, State: Initialized, Role: FOLLOWER
I20260809 08:45:13.940451 21209 consensus_queue.cc:260] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:13.940783 21196 heartbeater.cc:503] Master 127.31.63.126:37975 was elected leader, sending a full tablet report...
I20260809 08:45:13.940804 21209 ts_tablet_manager.cc:1428] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:45:13.941488 21210 raft_consensus.cc:348] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:13.941721 21210 raft_consensus.cc:374] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:13.941771 21210 raft_consensus.cc:729] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ae811764c94cf1ae024700ea2a8407, State: Initialized, Role: FOLLOWER
I20260809 08:45:13.941908 21210 consensus_queue.cc:260] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:13.942155 21210 ts_tablet_manager.cc:1428] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 08:45:13.942204 20994 heartbeater.cc:503] Master 127.31.63.126:37975 was elected leader, sending a full tablet report...
W20260809 08:45:14.007407 20995 tablet.cc:2369] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:45:14.028077 21214 raft_consensus.cc:482] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:14.028221 21214 raft_consensus.cc:504] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:14.028676 21214 leader_election.cc:288] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919), 5f3bd4b00fce4eb9800f05248c6e3987 (127.31.63.65:38159)
I20260809 08:45:14.031754 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.028992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:44933 (local address 127.31.63.65:38159)
0809 08:45:14.029056 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:14.029058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:14.029066 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:45:14.029102 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:14.029103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:14.029114 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:14.029184 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:14.029374 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:14.029953 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:14.031004 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:14.031385 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:14.031475 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:14.031485 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:14.031645 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 08:45:14.031687 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:45:14.031764 21200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.028882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:38159 (local address 127.31.63.67:44933)
0809 08:45:14.028950 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.028962 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:14.029008 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:14.029187 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:14.029189 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:14.029191 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:14.029294 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:14.029295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:14.030037 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:14.030039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:14.030910 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:14.030913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:14.031473 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:14.031474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:14.031567 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:14.031570 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:45:14.031639 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:45:14.031681 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 08:45:14.031895 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.028962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:49267 (local address 127.31.63.66:39919)
0809 08:45:14.029015 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:14.029018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:14.029181 (+ 163us) server_negotiation.cc:407] Connection header received
0809 08:45:14.029260 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:14.029261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:14.029272 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:14.029348 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:14.029552 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:14.030036 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:14.031109 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:14.031510 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:14.031608 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:14.031617 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:14.031762 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 08:45:14.031801 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:45:14.032115 21217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.028908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:39919 (local address 127.31.63.67:49267)
0809 08:45:14.029102 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.029112 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:14.029196 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:14.029353 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:14.029353 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:14.029355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:14.029443 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:14.029445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:14.030136 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:14.030137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:14.031000 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:14.031002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:14.031604 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:14.031605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:14.031684 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:14.031688 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:14.031757 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:45:14.031795 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":107,"threads_started":1}
I20260809 08:45:14.032063 20722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d26084767a57417e869414dd32522bfe" candidate_uuid: "978602d304e8405dad341e2335242eab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" is_pre_election: true
I20260809 08:45:14.032235 20722 raft_consensus.cc:2454] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978602d304e8405dad341e2335242eab in term 0.
I20260809 08:45:14.032533 21026 leader_election.cc:302] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3bd4b00fce4eb9800f05248c6e3987, 978602d304e8405dad341e2335242eab; no voters:
I20260809 08:45:14.032476 20932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d26084767a57417e869414dd32522bfe" candidate_uuid: "978602d304e8405dad341e2335242eab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46ae811764c94cf1ae024700ea2a8407" is_pre_election: true
I20260809 08:45:14.032665 20932 raft_consensus.cc:2454] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978602d304e8405dad341e2335242eab in term 0.
I20260809 08:45:14.032692 21214 raft_consensus.cc:2781] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:45:14.032763 21214 raft_consensus.cc:482] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:45:14.032793 21214 raft_consensus.cc:3037] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:14.033654 21214 raft_consensus.cc:504] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:14.033802 21214 leader_election.cc:288] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [CANDIDATE]: Term 1 election: Requested vote from peers 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919), 5f3bd4b00fce4eb9800f05248c6e3987 (127.31.63.65:38159)
I20260809 08:45:14.033996 20932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d26084767a57417e869414dd32522bfe" candidate_uuid: "978602d304e8405dad341e2335242eab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46ae811764c94cf1ae024700ea2a8407"
I20260809 08:45:14.034035 20722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d26084767a57417e869414dd32522bfe" candidate_uuid: "978602d304e8405dad341e2335242eab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3bd4b00fce4eb9800f05248c6e3987"
I20260809 08:45:14.034089 20932 raft_consensus.cc:3037] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:14.034149 20722 raft_consensus.cc:3037] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:14.035147 20932 raft_consensus.cc:2454] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978602d304e8405dad341e2335242eab in term 1.
I20260809 08:45:14.035368 21025 leader_election.cc:302] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46ae811764c94cf1ae024700ea2a8407, 978602d304e8405dad341e2335242eab; no voters:
I20260809 08:45:14.035493 21214 raft_consensus.cc:2781] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:45:14.035537 20722 raft_consensus.cc:2454] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978602d304e8405dad341e2335242eab in term 1.
I20260809 08:45:14.035691 21214 raft_consensus.cc:686] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 1 LEADER]: Becoming Leader. State: Replica: 978602d304e8405dad341e2335242eab, State: Running, Role: LEADER
I20260809 08:45:14.035811 21214 consensus_queue.cc:237] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:14.036763 20497 catalog_manager.cc:5495] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab reported cstate change: term changed from 0 to 1, leader changed from <none> to 978602d304e8405dad341e2335242eab (127.31.63.67). New cstate: current_term: 1 leader_uuid: "978602d304e8405dad341e2335242eab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } health_report { overall_health: UNKNOWN } } }
I20260809 08:45:14.048094 21206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.045740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33270 (local address 127.31.63.67:38847)
0809 08:45:14.045802 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:14.045804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:14.045835 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:45:14.045866 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:14.045867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:14.045884 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:14.045937 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:45:14.046055 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:14.046519 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:14.047372 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:14.047550 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:14.047599 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:14.047751 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:45:14.047845 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:45:14.047904 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:14.047958 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 08:45:14.047974 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:45:14.049940 20722 raft_consensus.cc:1261] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Refusing update from remote peer 978602d304e8405dad341e2335242eab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:14.049942 20932 raft_consensus.cc:1261] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407 [term 1 FOLLOWER]: Refusing update from remote peer 978602d304e8405dad341e2335242eab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:14.050341 21218 consensus_queue.cc:1035] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [LEADER]: Connected to new peer: Peer: permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:14.050489 21214 consensus_queue.cc:1035] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:14.055758 20497 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:45:14.055847 20497 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:45:14.056622 20497 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Sending DeleteTablet for 3 replicas of tablet d26084767a57417e869414dd32522bfe
I20260809 08:45:14.056939 21102 tablet_service.cc:1514] Processing DeleteTablet for tablet d26084767a57417e869414dd32522bfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:14 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:33268
I20260809 08:45:14.056967 20894 tablet_service.cc:1514] Processing DeleteTablet for tablet d26084767a57417e869414dd32522bfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:14 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:49858
I20260809 08:45:14.057251 21237 tablet_replica.cc:330] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: stopping tablet replica
I20260809 08:45:14.057333 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 20554
I20260809 08:45:14.057363 21237 raft_consensus.cc:2229] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:45:14.057348 20683 tablet_service.cc:1514] Processing DeleteTablet for tablet d26084767a57417e869414dd32522bfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:14 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:52174
I20260809 08:45:14.057426 20801 tablet_replica.cc:330] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: stopping tablet replica
I20260809 08:45:14.057473 20801 raft_consensus.cc:2229] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:45:14.057530 20801 raft_consensus.cc:2258] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:14.061211 21237 raft_consensus.cc:2258] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:14.062942 21237 ts_tablet_manager.cc:1905] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:14.064855 21237 ts_tablet_manager.cc:1918] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:14.064924 21237 log.cc:1196] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/d26084767a57417e869414dd32522bfe
I20260809 08:45:14.065184 21237 ts_tablet_manager.cc:1939] T d26084767a57417e869414dd32522bfe P 46ae811764c94cf1ae024700ea2a8407: Deleting consensus metadata
I20260809 08:45:14.065807 20465 catalog_manager.cc:4841] TS 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919): tablet d26084767a57417e869414dd32522bfe (table test-table [id=3d863463b6ec43aea4bbd1eb91b2780e]) successfully deleted
I20260809 08:45:14.073702 21236 tablet_replica.cc:330] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: stopping tablet replica
I20260809 08:45:14.073864 21236 raft_consensus.cc:2229] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:45:14.074028 21236 raft_consensus.cc:2258] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:14.074504 21236 ts_tablet_manager.cc:1905] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:14.075827 21236 ts_tablet_manager.cc:1918] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:14.075928 21236 log.cc:1196] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/d26084767a57417e869414dd32522bfe
I20260809 08:45:14.076206 21236 ts_tablet_manager.cc:1939] T d26084767a57417e869414dd32522bfe P 978602d304e8405dad341e2335242eab: Deleting consensus metadata
I20260809 08:45:14.076699 20464 catalog_manager.cc:4841] TS 978602d304e8405dad341e2335242eab (127.31.63.67:38847): tablet d26084767a57417e869414dd32522bfe (table test-table [id=3d863463b6ec43aea4bbd1eb91b2780e]) successfully deleted
W20260809 08:45:14.077165 20466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:38159 (error 108)
I20260809 08:45:14.077415 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 20795
I20260809 08:45:14.079656 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.077607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:38159 (local address 127.0.0.1:52180)
0809 08:45:14.079578 (+ 1971us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.079626 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:38159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1944}
W20260809 08:45:14.079924 20466 catalog_manager.cc:4584] TS 5f3bd4b00fce4eb9800f05248c6e3987 (127.31.63.65:38159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d26084767a57417e869414dd32522bfe: Network error: Client connection negotiation failed: client connection to 127.31.63.65:38159: connect: Connection refused (error 111)
I20260809 08:45:14.088088 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 20997
I20260809 08:45:14.104312 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 20431
I20260809 08:45:14.117997 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37975
--webserver_interface=127.31.63.126
--webserver_port=41029
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:37975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:14.203234 21242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:14.203464 21242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:14.203502 21242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:14.204819 21242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:14.204888 21242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:14.204913 21242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:14.204965 21242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:14.204994 21242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:14.206557 21242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:37975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=41029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:14.206841 21242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:14.207072 21242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:14.212309 21258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:14.212352 21256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:14.212309 21253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:14.212627 21242 server_base.cc:1034] running on GCE node
I20260809 08:45:14.212872 21242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:14.213227 21242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:14.214394 21242 hybrid_clock.cc:648] HybridClock initialized: now 1786265114214380 us; error 28 us; skew 500 ppm
Aug 09 08:45:14 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265114, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:14.217057 21242 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:45:14.217617 21242 webserver.cc:460] Webserver started at http://127.31.63.126:41029/ using document root <none> and password file <none>
I20260809 08:45:14.217866 21242 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:14.217913 21242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:14.219337 21242 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:14.220049 21270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:14.220268 21242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:14.220359 21242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "fcbad767f2244341a799a96daa358c1e"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "21459d89286fa2943a6c94b828579538"
server_key_iv: "e5ff227e3756564b194080d25e67bad6"
server_key_version: "encryptionkey@0"
I20260809 08:45:14.220700 21242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:14.235466 21242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:14.237829 21242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:14.237957 21242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:14.242165 21242 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:37975
I20260809 08:45:14.242204 21341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:37975 every 8 connection(s)
I20260809 08:45:14.242610 21242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:45:14.243721 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 21242
I20260809 08:45:14.244037 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:38159
--local_ip_for_outbound_sockets=127.31.63.65
--tserver_master_addrs=127.31.63.126:37975
--webserver_port=39155
--webserver_interface=127.31.63.65
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:14.244426 21342 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:45:14.245110 21342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Bootstrap starting.
I20260809 08:45:14.255208 21342 log.cc:824] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:14.257524 21342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:45:14.257901 21342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e: Bootstrap complete.
I20260809 08:45:14.260340 21342 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:14.260594 21342 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcbad767f2244341a799a96daa358c1e, State: Initialized, Role: FOLLOWER
I20260809 08:45:14.260819 21342 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:14.260921 21342 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:45:14.260977 21342 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:45:14.261025 21342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 2 FOLLOWER]: Advancing to term 3
I20260809 08:45:14.262140 21342 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:14.262276 21342 leader_election.cc:302] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcbad767f2244341a799a96daa358c1e; no voters:
I20260809 08:45:14.262483 21342 leader_election.cc:288] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 08:45:14.262523 21351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 3 FOLLOWER]: Leader election won for term 3
I20260809 08:45:14.262643 21351 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [term 3 LEADER]: Becoming Leader. State: Replica: fcbad767f2244341a799a96daa358c1e, State: Running, Role: LEADER
I20260809 08:45:14.262749 21351 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } }
I20260809 08:45:14.262777 21342 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:14.263036 21352 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fcbad767f2244341a799a96daa358c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } } }
I20260809 08:45:14.263134 21352 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:14.263298 21353 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcbad767f2244341a799a96daa358c1e. Latest consensus state: current_term: 3 leader_uuid: "fcbad767f2244341a799a96daa358c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcbad767f2244341a799a96daa358c1e" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37975 } } }
I20260809 08:45:14.263376 21353 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcbad767f2244341a799a96daa358c1e [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:14.263501 21356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:45:14.264345 21356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:45:14.264464 21356 catalog_manager.cc:1259] Loaded cluster ID: fa108a38c71948c2b5b6e37a394b4197
I20260809 08:45:14.264671 21356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 08:45:14.265177 21356 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb719e3c9ad google::LogMessage::Fail() at ??:0
@ 0x7fb719e3f88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb719e3c4de google::LogMessage::Flush() at ??:0
@ 0x7fb719e3da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb721d5d0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb721d668b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb71bd0a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb71bcff954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb71cb636db start_thread at ??:0
@ 0x7fb71893a61f clone at ??:0
W20260809 08:45:14.367910 21347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:14.368213 21347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:14.368256 21347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:14.370056 21347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:14.370122 21347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:14.370172 21347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:45:14.372361 21347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:38159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=39155
--tserver_master_addrs=127.31.63.126:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:14.372619 21347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:14.372896 21347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:14.378330 21394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:14.378377 21397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:14.378458 21393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:14.378716 21347 server_base.cc:1034] running on GCE node
I20260809 08:45:14.378937 21347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:14.379208 21347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:14.380406 21347 hybrid_clock.cc:648] HybridClock initialized: now 1786265114380392 us; error 29 us; skew 500 ppm
Aug 09 08:45:14 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265114, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:14.383960 21347 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:45:14.384518 21347 webserver.cc:460] Webserver started at http://127.31.63.65:39155/ using document root <none> and password file <none>
I20260809 08:45:14.384805 21347 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:14.384876 21347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:14.386742 21347 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:14.387660 21405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:14.387871 21347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:14.387977 21347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "5f3bd4b00fce4eb9800f05248c6e3987"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "7a9f28125836c6bf129722f2fc1496fd"
server_key_iv: "4c288b83df27453e0b2257d45681d8d7"
server_key_version: "encryptionkey@0"
I20260809 08:45:14.388312 21347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:14.397490 21347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:14.399796 21347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:14.399931 21347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:14.400173 21347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:14.400667 21418 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:45:14.402907 21347 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:45:14.402968 21347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:45:14.403026 21347 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:45:14.403847 21347 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:45:14.403901 21347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 08:45:14.403967 21418 tablet_bootstrap.cc:492] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: Bootstrap starting.
I20260809 08:45:14.411682 21347 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:38159
I20260809 08:45:14.412068 21347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:45:14.413700 21580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:38159 every 8 connection(s)
I20260809 08:45:14.417482 21418 log.cc:824] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:14.420459 21418 tablet_bootstrap.cc:492] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:45:14.420912 21418 tablet_bootstrap.cc:492] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: Bootstrap complete.
I20260809 08:45:14.421236 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 21347
I20260809 08:45:14.421495 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:39919
--local_ip_for_outbound_sockets=127.31.63.66
--tserver_master_addrs=127.31.63.126:37975
--webserver_port=34103
--webserver_interface=127.31.63.66
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:14.422605 21418 ts_tablet_manager.cc:1397] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.008s
I20260809 08:45:14.424237 21418 raft_consensus.cc:348] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:14.424703 21418 raft_consensus.cc:729] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3bd4b00fce4eb9800f05248c6e3987, State: Initialized, Role: FOLLOWER
I20260809 08:45:14.424912 21418 consensus_queue.cc:260] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:14.425231 21418 ts_tablet_manager.cc:1428] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
W20260809 08:45:14.509266 21586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:14.509511 21586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:14.509579 21586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:14.510762 21586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:14.510828 21586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:14.510874 21586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:45:14.512411 21586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:39919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=34103
--tserver_master_addrs=127.31.63.126:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:14.512682 21586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:14.512928 21586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:14.518325 21602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:14.518373 21595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:14.518478 21586 server_base.cc:1034] running on GCE node
W20260809 08:45:14.518607 21597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:14.518846 21586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:14.519109 21586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:14.520326 21586 hybrid_clock.cc:648] HybridClock initialized: now 1786265114520314 us; error 36 us; skew 500 ppm
Aug 09 08:45:14 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265114, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:14.523316 21586 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:45:14.523921 21586 webserver.cc:460] Webserver started at http://127.31.63.66:34103/ using document root <none> and password file <none>
I20260809 08:45:14.527936 21586 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:14.528019 21586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:14.530788 21586 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 08:45:14.531572 21612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:14.531880 21586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:45:14.531994 21586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "46ae811764c94cf1ae024700ea2a8407"
format_stamp: "Formatted at 2026-08-09 08:45:12 on dist-test-slave-gkw7"
server_key: "a447242ff26eafcbfd3640154488c15e"
server_key_iv: "a870aa2524e884ba5b2c309d041a342a"
server_key_version: "encryptionkey@0"
I20260809 08:45:14.532430 21586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260809 08:45:14.535845 21584 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:45:14.412859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:37311)
0809 08:45:14.413063 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.413108 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:14.413605 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:14.535615 (+122010us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":121,"threads_started":1}
W20260809 08:45:14.536137 21581 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:37975 (0 consecutive failures): Network error: Failed to ping master at 127.31.63.126:37975: Client connection negotiation failed: client connection to 127.31.63.126:37975: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 08:45:14.536497 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.536375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:59139)
0809 08:45:14.536431 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.536474 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 08:45:14.536878 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.536795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:54875)
0809 08:45:14.536827 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.536851 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 08:45:14.537020 21581 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:45:14.543372 21586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:14.546000 21586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:14.546205 21586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:14.546463 21586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:14.546797 21586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:14.546839 21586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:14.546864 21586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:14.546914 21586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:14.553807 21586 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:39919
I20260809 08:45:14.553871 21775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:39919 every 8 connection(s)
I20260809 08:45:14.554273 21586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:45:14.555405 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.554987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:49933)
0809 08:45:14.555173 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.555256 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
W20260809 08:45:14.555871 21776 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:37975 (0 consecutive failures): Network error: Failed to ping master at 127.31.63.126:37975: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
I20260809 08:45:14.556172 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.556076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:36365)
0809 08:45:14.556124 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.556151 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:45:14.556559 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.556478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:40099)
0809 08:45:14.556520 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.556543 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 08:45:14.556710 21776 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:45:14.559854 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 21586
I20260809 08:45:14.560057 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:38847
--local_ip_for_outbound_sockets=127.31.63.67
--tserver_master_addrs=127.31.63.126:37975
--webserver_port=39771
--webserver_interface=127.31.63.67
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:14.644901 21780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:14.645120 21780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:14.645144 21780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:14.646379 21780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:14.646426 21780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:14.646453 21780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:45:14.647935 21780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:38847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=39771
--tserver_master_addrs=127.31.63.126:37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:14.648145 21780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:14.648339 21780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:14.653664 21787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:14.653654 21788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:14.653658 21790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:14.653895 21780 server_base.cc:1034] running on GCE node
I20260809 08:45:14.654105 21780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:14.654382 21780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:14.655570 21780 hybrid_clock.cc:648] HybridClock initialized: now 1786265114655553 us; error 44 us; skew 500 ppm
Aug 09 08:45:14 dist-test-slave-gkw7 krb5kdc[19614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265114, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:14.658375 21780 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:45:14.658830 21780 webserver.cc:460] Webserver started at http://127.31.63.67:39771/ using document root <none> and password file <none>
I20260809 08:45:14.659054 21780 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:14.659099 21780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:14.660531 21780 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:14.661262 21798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:14.661489 21780 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:14.661631 21780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "978602d304e8405dad341e2335242eab"
format_stamp: "Formatted at 2026-08-09 08:45:13 on dist-test-slave-gkw7"
server_key: "7b031377afd986b52227180e0cec118f"
server_key_iv: "fb1a1306ee04767de2ad1ed4158eeca6"
server_key_version: "encryptionkey@0"
I20260809 08:45:14.662009 21780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:14.674639 21780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:14.676710 21780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:14.676826 21780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:14.677026 21780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:14.677363 21780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:14.677398 21780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:14.677421 21780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:14.677472 21780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:14.683732 21780 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:38847
I20260809 08:45:14.683809 21954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:38847 every 8 connection(s)
I20260809 08:45:14.684180 21780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:45:14.685369 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.684934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:34991)
0809 08:45:14.685128 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.685221 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
W20260809 08:45:14.685842 21955 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:37975 (0 consecutive failures): Network error: Failed to ping master at 127.31.63.126:37975: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
I20260809 08:45:14.686039 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 21780
I20260809 08:45:14.686164 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.686040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:39171)
0809 08:45:14.686093 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.686120 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 08:45:14.686463 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:14.686373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:45631)
0809 08:45:14.686418 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:14.686441 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 08:45:14.686632 21955 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:45:15.537654 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:15.537398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:48243)
0809 08:45:15.537518 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:15.537583 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260809 08:45:15.557278 21961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:15.557043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:60905)
0809 08:45:15.557189 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:15.557222 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":105,"threads_started":1}
I20260809 08:45:15.687176 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:15.686942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:36551)
0809 08:45:15.687075 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:15.687116 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":58,"threads_started":1}
I20260809 08:45:15.803917 21966 raft_consensus.cc:482] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:15.804095 21966 raft_consensus.cc:504] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:15.804531 21966 leader_election.cc:288] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978602d304e8405dad341e2335242eab (127.31.63.67:38847), 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919)
Aug 09 08:45:15 dist-test-slave-gkw7 krb5kdc[19614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265114, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.66@KRBTEST.COM
Aug 09 08:45:15 dist-test-slave-gkw7 krb5kdc[19614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265114, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.67@KRBTEST.COM
I20260809 08:45:15.815551 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:15.804729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:39919 (local address 127.31.63.65:59547)
0809 08:45:15.804866 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:15.804876 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:15.804934 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:15.805817 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:15.805819 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:15.806237 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:15.806421 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:15.806427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:15.807536 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:15.807537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:15.808530 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:15.808533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:15.809133 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:15.809134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:15.809274 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:15.809950 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:15.809959 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:15.811590 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:15.813764 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:15.813771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:15.813774 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:15.814062 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:15.814355 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:15.814356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:15.814356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:15.814455 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:15.814972 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:15.814980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:15.815128 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 08:45:15.815237 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 08:45:15.815441 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":13,"thread_start_us":60,"threads_started":1}
I20260809 08:45:15.815603 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:15.804859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:59547 (local address 127.31.63.66:39919)
0809 08:45:15.805025 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:15.805035 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:15.805066 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:45:15.805626 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:15.805630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:15.805670 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:15.805827 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:15.806554 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:15.807428 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:15.808620 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:15.809035 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:15.809162 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:15.811698 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:15.811703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:15.811703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:15.811728 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:15.813680 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:15.814147 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:15.814150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:15.814150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:15.814239 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:15.814524 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:15.814525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:15.814525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:15.814895 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:15.814965 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:15.815311 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 08:45:15.815461 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260809 08:45:15.815698 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:15.804807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:50029 (local address 127.31.63.67:38847)
0809 08:45:15.805008 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:15.805027 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:15.805056 (+ 29us) server_negotiation.cc:407] Connection header received
0809 08:45:15.805523 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:15.805525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:15.805629 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:15.805781 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:15.806553 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:15.807429 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:15.808621 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:15.809035 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:15.809171 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:15.812156 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:15.812160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:15.812160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:15.812172 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:15.814209 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:15.814593 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:15.814596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:15.814596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:15.814672 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:15.814870 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:15.814871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:15.814871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:15.815148 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:15.815259 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:15.815453 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 08:45:15.815579 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":79,"threads_started":1}
I20260809 08:45:15.816329 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:15.804675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:38847 (local address 127.31.63.65:50029)
0809 08:45:15.804743 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:15.804757 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:15.804822 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:15.805813 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:15.805815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:15.806235 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:15.806421 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:15.806427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:15.807577 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:15.807578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:15.808529 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:15.808532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:15.809142 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:15.809143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:15.809277 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:15.810004 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:15.810008 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:15.812043 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:15.814287 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:15.814288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:15.814289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:15.814473 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:15.814740 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:15.814741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:15.814741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:15.814803 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:15.815249 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:15.815250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:15.815296 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 08:45:15.815374 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:45:15.815525 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":157}
W20260809 08:45:15.816694 21408 leader_election.cc:341] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919): Not found: Tablet not found: d26084767a57417e869414dd32522bfe
W20260809 08:45:15.817090 21407 leader_election.cc:341] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 978602d304e8405dad341e2335242eab (127.31.63.67:38847): Not found: Tablet not found: d26084767a57417e869414dd32522bfe
I20260809 08:45:15.817149 21407 leader_election.cc:302] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f3bd4b00fce4eb9800f05248c6e3987; no voters: 46ae811764c94cf1ae024700ea2a8407, 978602d304e8405dad341e2335242eab
I20260809 08:45:15.817250 21966 raft_consensus.cc:2726] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:45:16.538290 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:16.538081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:51109)
0809 08:45:16.538207 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:16.538241 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260809 08:45:16.557952 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:16.557752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:59709)
0809 08:45:16.557877 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:16.557913 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260809 08:45:16.687852 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:16.687656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:41349)
0809 08:45:16.687776 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:16.687812 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260809 08:45:17.539032 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:17.538810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:51617)
0809 08:45:17.538946 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:17.538985 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260809 08:45:17.558705 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:17.558447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:60275)
0809 08:45:17.558620 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:17.558661 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":133,"threads_started":1}
I20260809 08:45:17.633808 21980 raft_consensus.cc:482] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:17.633951 21980 raft_consensus.cc:504] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:17.634193 21980 leader_election.cc:288] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978602d304e8405dad341e2335242eab (127.31.63.67:38847), 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919)
W20260809 08:45:17.634717 21408 leader_election.cc:341] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919): Not found: Tablet not found: d26084767a57417e869414dd32522bfe
W20260809 08:45:17.634805 21407 leader_election.cc:341] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 978602d304e8405dad341e2335242eab (127.31.63.67:38847): Not found: Tablet not found: d26084767a57417e869414dd32522bfe
I20260809 08:45:17.634909 21407 leader_election.cc:302] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f3bd4b00fce4eb9800f05248c6e3987; no voters: 46ae811764c94cf1ae024700ea2a8407, 978602d304e8405dad341e2335242eab
I20260809 08:45:17.635000 21980 raft_consensus.cc:2726] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:45:17.688513 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:17.688272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:52277)
0809 08:45:17.688429 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:17.688470 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260809 08:45:18.539801 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:18.539575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:59461)
0809 08:45:18.539704 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:18.539742 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260809 08:45:18.559391 21984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:18.559187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:44859)
0809 08:45:18.559324 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:18.559354 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260809 08:45:18.689249 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:18.689021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:44611)
0809 08:45:18.689170 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:18.689204 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260809 08:45:19.540592 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:19.540323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:46767)
0809 08:45:19.540484 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:19.540527 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
I20260809 08:45:19.560102 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:19.559862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:34695)
0809 08:45:19.560025 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:19.560057 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260809 08:45:19.689975 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:19.689744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:60617)
0809 08:45:19.689891 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:19.689933 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260809 08:45:20.173281 21993 raft_consensus.cc:482] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:20.173395 21993 raft_consensus.cc:504] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:20.173624 21993 leader_election.cc:288] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978602d304e8405dad341e2335242eab (127.31.63.67:38847), 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919)
W20260809 08:45:20.174021 21408 leader_election.cc:341] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919): Not found: Tablet not found: d26084767a57417e869414dd32522bfe
W20260809 08:45:20.174173 21407 leader_election.cc:341] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 978602d304e8405dad341e2335242eab (127.31.63.67:38847): Not found: Tablet not found: d26084767a57417e869414dd32522bfe
I20260809 08:45:20.174252 21407 leader_election.cc:302] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f3bd4b00fce4eb9800f05248c6e3987; no voters: 46ae811764c94cf1ae024700ea2a8407, 978602d304e8405dad341e2335242eab
I20260809 08:45:20.174365 21993 raft_consensus.cc:2726] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:45:20.541438 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:20.541143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:35571)
0809 08:45:20.541337 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:20.541374 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":118,"threads_started":1}
I20260809 08:45:20.560822 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:20.560599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:47549)
0809 08:45:20.560737 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:20.560778 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260809 08:45:20.690747 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:20.690506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:59731)
0809 08:45:20.690638 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:20.690680 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260809 08:45:21.542341 21999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:21.542053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:40427)
0809 08:45:21.542247 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:21.542291 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":106,"threads_started":1}
I20260809 08:45:21.561515 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:21.561299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:47441)
0809 08:45:21.561431 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:21.561471 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260809 08:45:21.691532 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:21.691283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:43305)
0809 08:45:21.691448 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:21.691485 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":68,"threads_started":1}
I20260809 08:45:22.368464 22002 raft_consensus.cc:482] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:22.368575 22002 raft_consensus.cc:504] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:22.368825 22002 leader_election.cc:288] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978602d304e8405dad341e2335242eab (127.31.63.67:38847), 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919)
W20260809 08:45:22.369284 21407 leader_election.cc:341] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 978602d304e8405dad341e2335242eab (127.31.63.67:38847): Not found: Tablet not found: d26084767a57417e869414dd32522bfe
W20260809 08:45:22.369366 21408 leader_election.cc:341] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919): Not found: Tablet not found: d26084767a57417e869414dd32522bfe
I20260809 08:45:22.369462 21408 leader_election.cc:302] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f3bd4b00fce4eb9800f05248c6e3987; no voters: 46ae811764c94cf1ae024700ea2a8407, 978602d304e8405dad341e2335242eab
I20260809 08:45:22.369575 22002 raft_consensus.cc:2726] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:45:22.543244 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:22.542891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:42091)
0809 08:45:22.543088 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:22.543143 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":73,"threads_started":1}
I20260809 08:45:22.562311 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:22.562079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:53133)
0809 08:45:22.562219 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:22.562259 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260809 08:45:22.692212 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:22.691987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:45793)
0809 08:45:22.692130 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:22.692167 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":63,"threads_started":1}
I20260809 08:45:23.543989 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:23.543756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:58109)
0809 08:45:23.543896 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:23.543938 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260809 08:45:23.563130 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:23.562841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:47703)
0809 08:45:23.563006 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:23.563054 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260809 08:45:23.692917 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:23.692671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:39015)
0809 08:45:23.692832 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:23.692869 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
I20260809 08:45:24.544807 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:24.544530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:48821)
0809 08:45:24.544720 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:24.544756 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":118,"threads_started":1}
I20260809 08:45:24.564020 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:24.563736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:57669)
0809 08:45:24.563920 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:24.563967 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":75,"threads_started":1}
I20260809 08:45:24.693840 22014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:24.693495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:51819)
0809 08:45:24.693737 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:24.693781 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":152,"threads_started":1}
I20260809 08:45:25.545665 22018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:25.545351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:48433)
0809 08:45:25.545508 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:25.545606 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260809 08:45:25.564778 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:25.564532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:40567)
0809 08:45:25.564692 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:25.564732 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":67,"threads_started":1}
I20260809 08:45:25.694655 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:25.694390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:58209)
0809 08:45:25.694558 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:25.694604 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":86,"threads_started":1}
I20260809 08:45:26.546358 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:26.546129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:40931)
0809 08:45:26.546269 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:26.546309 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260809 08:45:26.565642 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:26.565336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:35161)
0809 08:45:26.565477 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:26.565568 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260809 08:45:26.695358 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:26.695122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:39103)
0809 08:45:26.695278 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:26.695314 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260809 08:45:27.429183 22027 raft_consensus.cc:482] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:27.429374 22027 raft_consensus.cc:504] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "978602d304e8405dad341e2335242eab" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 38847 } } peers { permanent_uuid: "46ae811764c94cf1ae024700ea2a8407" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 39919 } } peers { permanent_uuid: "5f3bd4b00fce4eb9800f05248c6e3987" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 38159 } }
I20260809 08:45:27.429644 22027 leader_election.cc:288] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 978602d304e8405dad341e2335242eab (127.31.63.67:38847), 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919)
W20260809 08:45:27.430085 21408 leader_election.cc:341] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 46ae811764c94cf1ae024700ea2a8407 (127.31.63.66:39919): Not found: Tablet not found: d26084767a57417e869414dd32522bfe
W20260809 08:45:27.430207 21407 leader_election.cc:341] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 978602d304e8405dad341e2335242eab (127.31.63.67:38847): Not found: Tablet not found: d26084767a57417e869414dd32522bfe
I20260809 08:45:27.430250 21407 leader_election.cc:302] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f3bd4b00fce4eb9800f05248c6e3987; no voters: 46ae811764c94cf1ae024700ea2a8407, 978602d304e8405dad341e2335242eab
I20260809 08:45:27.430379 22027 raft_consensus.cc:2726] T d26084767a57417e869414dd32522bfe P 5f3bd4b00fce4eb9800f05248c6e3987 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:45:27.547317 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:27.546999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:45933)
0809 08:45:27.547217 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:27.547268 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
I20260809 08:45:27.566452 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:27.566217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:49087)
0809 08:45:27.566362 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:27.566407 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260809 08:45:27.696193 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:27.695926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:34091)
0809 08:45:27.696086 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:27.696138 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":66,"threads_started":1}
I20260809 08:45:28.548280 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:28.547906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:37513)
0809 08:45:28.548160 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:28.548220 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":97,"threads_started":1}
I20260809 08:45:28.567343 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:28.567039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:51551)
0809 08:45:28.567227 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:28.567291 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260809 08:45:28.697045 22037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:28.696760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:46797)
0809 08:45:28.696942 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:28.696994 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260809 08:45:29.549342 22038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:29.549056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.65:55333)
0809 08:45:29.549226 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:29.549292 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":80,"threads_started":1}
I20260809 08:45:29.568217 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:29.567980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.66:46145)
0809 08:45:29.568131 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:29.568171 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260809 08:45:29.697464 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 21347
I20260809 08:45:29.700114 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:29.697731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37975 (local address 127.31.63.67:52371)
0809 08:45:29.699982 (+ 2251us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:29.700039 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2224,"thread_start_us":97,"threads_started":1}
I20260809 08:45:29.709292 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 21586
I20260809 08:45:29.719765 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 21780
2026-08-09T08:45:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18889 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:45:30 dist-test-slave-gkw7 krb5kdc[22047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:45:30 dist-test-slave-gkw7 krb5kdc[22047](info): set up 2 sockets
Aug 09 08:45:30 dist-test-slave-gkw7 krb5kdc[22047](info): commencing operation
krb5kdc: starting...
W20260809 08:45:31.283757 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:45:31 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:45:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:45:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:45:31.413422 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37669
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:37669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:31.498000 22064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:31.498277 22064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:31.498317 22064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:31.499503 22064 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 08:45:31.499580 22064 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 08:45:31.499604 22064 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 08:45:31.499624 22064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:31.499655 22064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:31.499683 22064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:31.499703 22064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:31.499725 22064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:31.501261 22064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:37669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:31.501576 22064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:31.501840 22064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:31.507440 22073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:31.507483 22075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:31.507519 22064 server_base.cc:1034] running on GCE node
W20260809 08:45:31.507498 22071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:31.507913 22064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:31.508219 22064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:31.509366 22064 hybrid_clock.cc:648] HybridClock initialized: now 1786265131509335 us; error 49 us; skew 500 ppm
Aug 09 08:45:31 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265131, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:31.511855 22064 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:45:31.512336 22064 webserver.cc:460] Webserver started at http://127.31.63.126:45731/ using document root <none> and password file <none>
I20260809 08:45:31.512570 22064 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:31.512622 22064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:31.512774 22064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:31.514163 22064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "50373fff1d50469188bfd92cde0a8611"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "e044627a74c00856825476dedd90f232"
server_key_iv: "c4cc3c54aa323341b0e120685f128562"
server_key_version: "encryptionkey@0"
I20260809 08:45:31.514549 22064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "50373fff1d50469188bfd92cde0a8611"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "e044627a74c00856825476dedd90f232"
server_key_iv: "c4cc3c54aa323341b0e120685f128562"
server_key_version: "encryptionkey@0"
I20260809 08:45:31.515949 22064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:45:31.516772 22082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:31.516986 22064 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:31.517093 22064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "50373fff1d50469188bfd92cde0a8611"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "e044627a74c00856825476dedd90f232"
server_key_iv: "c4cc3c54aa323341b0e120685f128562"
server_key_version: "encryptionkey@0"
I20260809 08:45:31.517207 22064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:31.535851 22064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:31.538157 22064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:31.538357 22064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:31.543066 22064 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:37669
I20260809 08:45:31.543102 22154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:37669 every 8 connection(s)
I20260809 08:45:31.543501 22064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:45:31.544443 22155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:31.546918 22155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Bootstrap starting.
I20260809 08:45:31.547590 22155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:31.547904 22155 log.cc:824] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:31.548707 22155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: No bootstrap required, opened a new log
I20260809 08:45:31.549466 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 22064
I20260809 08:45:31.549633 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:45:31.549842 31997 external_mini_cluster.cc:1428] Setting key ca6e48505eea227ca87e5cf4f7bad818
I20260809 08:45:31.550817 22155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:45:31.550990 22155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:31.551082 22155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50373fff1d50469188bfd92cde0a8611, State: Initialized, Role: FOLLOWER
I20260809 08:45:31.551266 22155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:45:31.551365 22155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:45:31.551412 22155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:45:31.551466 22155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:31.552269 22155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:45:31.552398 22155 leader_election.cc:302] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50373fff1d50469188bfd92cde0a8611; no voters:
I20260809 08:45:31.552611 22155 leader_election.cc:288] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:45:31.552649 22163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:45:31.552759 22163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 1 LEADER]: Becoming Leader. State: Replica: 50373fff1d50469188bfd92cde0a8611, State: Running, Role: LEADER
I20260809 08:45:31.552891 22155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:31.552906 22163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:45:31.553748 22165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50373fff1d50469188bfd92cde0a8611. Latest consensus state: current_term: 1 leader_uuid: "50373fff1d50469188bfd92cde0a8611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } } }
I20260809 08:45:31.553910 22165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:31.554179 22175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:45:31.554708 22175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:45:31.554678 22164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50373fff1d50469188bfd92cde0a8611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } } }
I20260809 08:45:31.555335 22164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: This master's current role is: LEADER
Aug 09 08:45:31 dist-test-slave-gkw7 krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:31.557313 22175 catalog_manager.cc:1347] Generated new cluster ID: 2cf68c810c9d4a4d8a2ebe605a6aca14
I20260809 08:45:31.557359 22175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:45:31.561343 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:31.550427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.31.63.126:37669)
0809 08:45:31.550648 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:31.550656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:31.550682 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:45:31.550782 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:31.550787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:31.550830 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:31.550979 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:31.551787 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:31.552916 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:31.554146 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:31.554748 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:31.554856 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:31.557903 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:31.557907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:31.557907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:31.557919 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:31.559724 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:31.560115 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:31.560118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:31.560119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:31.560233 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:31.560457 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:31.560458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:31.560459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:31.560714 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:31.560872 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:31.560988 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 08:45:31.561149 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260809 08:45:31.577035 22175 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:45:31.577858 22175 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:45:31.586787 22175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Generated new TSK 0
I20260809 08:45:31.587024 22175 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:45:31.621284 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37669
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:31.707942 22188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:31.708155 22188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:31.708179 22188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:31.709336 22188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:31.709381 22188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:31.709450 22188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:45:31.710994 22188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:31.711243 22188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:31.711480 22188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:31.716612 22194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:31.716645 22197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:31.716681 22199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:31.716874 22188 server_base.cc:1034] running on GCE node
I20260809 08:45:31.717059 22188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:31.717252 22188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:31.718393 22188 hybrid_clock.cc:648] HybridClock initialized: now 1786265131718380 us; error 28 us; skew 500 ppm
Aug 09 08:45:31 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265131, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:31.721045 22188 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:45:31.721632 22188 webserver.cc:460] Webserver started at http://127.31.63.65:45639/ using document root <none> and password file <none>
I20260809 08:45:31.721896 22188 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:31.721946 22188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:31.722101 22188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:31.723157 22188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "ac207fc59f2644c086ecb956acf15f19"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "03c3e72c85968851b7a34b5807949b7d"
server_key_iv: "89849320ff0c6074469874f54f0f0b13"
server_key_version: "encryptionkey@0"
I20260809 08:45:31.723603 22188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "ac207fc59f2644c086ecb956acf15f19"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "03c3e72c85968851b7a34b5807949b7d"
server_key_iv: "89849320ff0c6074469874f54f0f0b13"
server_key_version: "encryptionkey@0"
I20260809 08:45:31.725021 22188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:45:31.725914 22209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:31.726096 22188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:45:31.726207 22188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "ac207fc59f2644c086ecb956acf15f19"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "03c3e72c85968851b7a34b5807949b7d"
server_key_iv: "89849320ff0c6074469874f54f0f0b13"
server_key_version: "encryptionkey@0"
I20260809 08:45:31.726295 22188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:31.745438 22188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:31.747723 22188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:31.747916 22188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:31.748170 22188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:31.748509 22188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:31.748569 22188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:31.748610 22188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:31.748638 22188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:31.755362 22188 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:41003
I20260809 08:45:31.755432 22361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:41003 every 8 connection(s)
I20260809 08:45:31.755796 22188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:45:31.757386 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 22188
I20260809 08:45:31.757534 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:45:31.757800 31997 external_mini_cluster.cc:1428] Setting key 29e9cd06afbca27b9d8961722dbeb157
Aug 09 08:45:31 dist-test-slave-gkw7 krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265131, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:31.766911 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:31.756584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:45635)
0809 08:45:31.756753 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:31.756805 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:31.757292 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:31.757578 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:31.757582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:31.758373 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:31.758647 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:31.758652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:31.759376 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:31.759377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:31.760189 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:31.760191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:31.760620 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:31.760621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:31.760789 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:31.761353 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:31.761364 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:31.763449 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:31.765433 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:31.765440 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:31.765442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:31.765746 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:31.765975 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:31.765976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:31.765976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:31.766077 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:31.766409 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:31.766414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:31.766533 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 08:45:31.766611 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:45:31.766738 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260809 08:45:31.766908 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:31.756668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:45635 (local address 127.31.63.126:37669)
0809 08:45:31.756719 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:31.756721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:31.756898 (+ 177us) server_negotiation.cc:407] Connection header received
0809 08:45:31.757409 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:31.757411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:31.757424 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:31.757481 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:31.758744 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:31.759262 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:31.760294 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:31.760526 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:31.760610 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:31.763548 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:31.763549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:31.763550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:31.763559 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:31.765341 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:31.765827 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:31.765828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:31.765829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:31.765898 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:31.766160 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:31.766162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:31.766162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:31.766338 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:31.766416 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:31.766685 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 08:45:31.766793 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:45:31.767560 22362 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37669
I20260809 08:45:31.767676 22362 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:31.767889 22362 heartbeater.cc:511] Master 127.31.63.126:37669 requested a full tablet report, sending...
I20260809 08:45:31.768474 22102 ts_manager.cc:194] Registered new tserver with Master: ac207fc59f2644c086ecb956acf15f19 (127.31.63.65:41003)
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
I20260809 08:45:31.769080 22102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:45635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:45:31.788947 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37669
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:31.872794 22368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:31.873050 22368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:31.873090 22368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:31.874271 22368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:31.874354 22368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:31.874399 22368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:45:31.875921 22368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:31.876201 22368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:31.876446 22368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:31.881770 22377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:31.881747 22375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:31.881737 22374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:31.881968 22368 server_base.cc:1034] running on GCE node
I20260809 08:45:31.882241 22368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:31.882452 22368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:31.883589 22368 hybrid_clock.cc:648] HybridClock initialized: now 1786265131883583 us; error 34 us; skew 500 ppm
Aug 09 08:45:31 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265131, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:31.886040 22368 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:45:31.886514 22368 webserver.cc:460] Webserver started at http://127.31.63.66:36941/ using document root <none> and password file <none>
I20260809 08:45:31.886752 22368 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:31.886829 22368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:31.886986 22368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:31.888033 22368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "b636bf8d5f9a458f926f8936b6ebe054"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "12233195af3fcc5b329888c99b677622"
server_key_iv: "06b822c23c5654febc27538aecac2eca"
server_key_version: "encryptionkey@0"
I20260809 08:45:31.888409 22368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "b636bf8d5f9a458f926f8936b6ebe054"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "12233195af3fcc5b329888c99b677622"
server_key_iv: "06b822c23c5654febc27538aecac2eca"
server_key_version: "encryptionkey@0"
I20260809 08:45:31.889910 22368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:45:31.890699 22388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:31.890938 22368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:31.891047 22368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "b636bf8d5f9a458f926f8936b6ebe054"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "12233195af3fcc5b329888c99b677622"
server_key_iv: "06b822c23c5654febc27538aecac2eca"
server_key_version: "encryptionkey@0"
I20260809 08:45:31.891126 22368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:31.907243 22368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:31.909176 22368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:31.909349 22368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:31.909646 22368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:31.909977 22368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:31.910035 22368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:31.910079 22368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:31.910106 22368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:31.916534 22368 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:41367
I20260809 08:45:31.916584 22536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:41367 every 8 connection(s)
I20260809 08:45:31.916908 22368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:45:31 dist-test-slave-gkw7 krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265131, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:31.924787 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 22368
I20260809 08:45:31.924901 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:45:31.925117 31997 external_mini_cluster.cc:1428] Setting key 38091bbf8515e67118b2a2e3b14d5c08
I20260809 08:45:31.926580 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:31.917711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:52603 (local address 127.31.63.126:37669)
0809 08:45:31.917809 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:31.917812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:31.917996 (+ 184us) server_negotiation.cc:407] Connection header received
0809 08:45:31.918496 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:31.918497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:31.918507 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:31.918560 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:31.919479 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:31.920078 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:31.921161 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:31.921351 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:31.921406 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:31.923655 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:31.923657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:31.923657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:31.923663 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:31.925074 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:31.925522 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:31.925523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:31.925523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:31.925631 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:31.925910 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:31.925911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:31.925911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:31.926038 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:31.926119 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:31.926399 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 08:45:31.926490 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 08:45:31.926772 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:31.917688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:52603)
0809 08:45:31.917872 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:31.917911 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:31.918412 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:31.918615 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:31.918618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:31.919204 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:31.919402 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:31.919407 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:31.920188 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:31.920189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:31.921084 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:31.921087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:31.921432 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:31.921433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:31.921585 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:31.922201 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:31.922209 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:31.923566 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:31.925179 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:31.925187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:31.925190 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:31.925433 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:31.925698 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:31.925699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:31.925699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:31.925832 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:31.926119 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:31.926127 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:31.926257 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 08:45:31.926366 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 08:45:31.926528 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
I20260809 08:45:31.927273 22537 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37669
I20260809 08:45:31.927385 22537 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:31.927565 22537 heartbeater.cc:511] Master 127.31.63.126:37669 requested a full tablet report, sending...
I20260809 08:45:31.927975 22102 ts_manager.cc:194] Registered new tserver with Master: b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367)
I20260809 08:45:31.928361 22102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:52603
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:45:31.953512 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37669
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:32.039518 22543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:32.039781 22543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:32.039824 22543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:32.041013 22543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:32.041088 22543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:32.041136 22543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:45:32.042732 22543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:37669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:32.043013 22543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:32.043277 22543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:32.048662 22557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:32.048667 22553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:32.048759 22543 server_base.cc:1034] running on GCE node
W20260809 08:45:32.048668 22554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:32.049202 22543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:32.049475 22543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:32.050665 22543 hybrid_clock.cc:648] HybridClock initialized: now 1786265132050652 us; error 30 us; skew 500 ppm
Aug 09 08:45:32 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265132, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:32.053455 22543 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:45:32.054040 22543 webserver.cc:460] Webserver started at http://127.31.63.67:38493/ using document root <none> and password file <none>
I20260809 08:45:32.054261 22543 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:32.054342 22543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:32.054455 22543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:45:32.055509 22543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "5df8657b7106472a98abc8cdeffdc431"
format_stamp: "Formatted at 2026-08-09 08:45:32 on dist-test-slave-gkw7"
server_key: "6343a7d2e72139f1d3edec0e9373660e"
server_key_iv: "2b515e23e65f5240599f8132d1b7a0d0"
server_key_version: "encryptionkey@0"
I20260809 08:45:32.055843 22543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "5df8657b7106472a98abc8cdeffdc431"
format_stamp: "Formatted at 2026-08-09 08:45:32 on dist-test-slave-gkw7"
server_key: "6343a7d2e72139f1d3edec0e9373660e"
server_key_iv: "2b515e23e65f5240599f8132d1b7a0d0"
server_key_version: "encryptionkey@0"
I20260809 08:45:32.057300 22543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:45:32.058235 22565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:32.058456 22543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:45:32.058579 22543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "5df8657b7106472a98abc8cdeffdc431"
format_stamp: "Formatted at 2026-08-09 08:45:32 on dist-test-slave-gkw7"
server_key: "6343a7d2e72139f1d3edec0e9373660e"
server_key_iv: "2b515e23e65f5240599f8132d1b7a0d0"
server_key_version: "encryptionkey@0"
I20260809 08:45:32.058667 22543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:32.072237 22543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:32.074805 22543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:32.074988 22543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:32.075243 22543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:32.075575 22543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:32.075632 22543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:32.075676 22543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:32.075703 22543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:32.081990 22543 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:40339
I20260809 08:45:32.082074 22744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:40339 every 8 connection(s)
I20260809 08:45:32.082413 22543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:45:32 dist-test-slave-gkw7 krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265132, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:32.089686 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 22543
I20260809 08:45:32.089792 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:45:32.090024 31997 external_mini_cluster.cc:1428] Setting key 49698df8cd0b13dbf9c7c624b9594c24
I20260809 08:45:32.094025 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.083264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:59253 (local address 127.31.63.126:37669)
0809 08:45:32.083353 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:32.083356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:32.083564 (+ 208us) server_negotiation.cc:407] Connection header received
0809 08:45:32.084216 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:32.084218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:32.084233 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:32.084291 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:32.085392 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.086036 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.087320 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.087558 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.087650 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.090690 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:32.090691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:32.090692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:32.090701 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:32.092401 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:32.092860 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:32.092862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:32.092862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:32.092926 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:32.093203 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:32.093205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:32.093205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:32.093334 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:32.093408 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:32.093803 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 08:45:32.093918 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:45:32.094269 22747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.083214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:59253)
0809 08:45:32.083431 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:32.083471 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:32.084060 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:32.084376 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:32.084380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:32.085049 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:32.085265 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.085270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.086149 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.086151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.087163 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.087166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.087661 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.087662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.087810 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.088336 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:32.088345 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:32.090469 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:32.092511 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:32.092520 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:32.092523 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:32.092767 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:32.093005 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:32.093006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:32.093006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:32.093124 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:32.093412 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:32.093420 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:32.093646 (+ 226us) client_negotiation.cc:769] Sending connection context
0809 08:45:32.093759 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 08:45:32.094006 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260809 08:45:32.094728 22745 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37669
I20260809 08:45:32.094854 22745 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:32.095117 22745 heartbeater.cc:511] Master 127.31.63.126:37669 requested a full tablet report, sending...
I20260809 08:45:32.095537 22102 ts_manager.cc:194] Registered new tserver with Master: 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339)
I20260809 08:45:32.095937 22102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:59253
I20260809 08:45:32.102214 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:45:32.110206 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.103772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.31.63.126:37669)
0809 08:45:32.103850 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:32.103853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:32.103937 (+ 84us) server_negotiation.cc:407] Connection header received
0809 08:45:32.103973 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:32.103974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:32.103984 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:32.104040 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:32.104647 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.105175 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.105985 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.106205 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.106271 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.107596 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:32.107598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:32.107598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:32.107604 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:32.108958 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:32.109302 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:32.109304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:32.109304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:32.109367 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:32.109607 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:32.109608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:32.109608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:32.109795 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:32.109890 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:32.110052 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 08:45:32.110129 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 08:45:32.111927 22102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:45:32.113858 22102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:45:32.120585 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.116615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.31.63.65:41003)
0809 08:45:32.116850 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:32.116856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:32.116871 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:45:32.116938 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:32.116940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:32.116959 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:32.117056 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:32.117267 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.118254 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.119380 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.120122 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.120199 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.120269 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:32.120462 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 08:45:32.120503 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":90,"threads_started":1}
I20260809 08:45:32.120579 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.116490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:41003 (local address 127.0.0.1:58244)
0809 08:45:32.116700 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:32.116748 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:32.116825 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:32.117080 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:32.117083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:32.117086 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:32.117178 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.117181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.118363 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.118364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.119281 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.119283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.120212 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.120213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.120327 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.120344 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 08:45:32.120431 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 08:45:32.120487 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"spinlock_wait_cycles":896,"thread_start_us":116,"threads_started":1}
I20260809 08:45:32.121644 22274 tablet_service.cc:1467] Processing CreateTablet for tablet f8513f2ff47d4180aab1b469a8c88df0 (DEFAULT_TABLE table=test-table [id=08c9db7202914d23bbc3f7ba11af8a07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:32.122035 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.116726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:41367 (local address 127.0.0.1:59628)
0809 08:45:32.117153 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:32.117162 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:32.117224 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:32.117985 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:32.117986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:32.117988 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:32.118030 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.118031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.119442 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.119443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.120695 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.120699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.121756 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.121757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.121847 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.121852 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:45:32.121890 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 08:45:32.121944 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":91,"threads_started":1}
I20260809 08:45:32.122153 22274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8513f2ff47d4180aab1b469a8c88df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:32.122044 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.116490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40339 (local address 127.0.0.1:43464)
0809 08:45:32.116990 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:32.116999 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:32.117068 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:32.117633 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:32.117634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:32.117636 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:32.117685 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.117687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.119153 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.119154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.120232 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.120235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.121815 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.121816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.121877 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.121881 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:32.121937 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 08:45:32.121970 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260809 08:45:32.122258 22763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.116696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43464 (local address 127.31.63.67:40339)
0809 08:45:32.117383 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:32.117388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:32.117399 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:45:32.117441 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:32.117442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:32.117460 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:32.117637 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:32.117787 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.119026 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.120326 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.121576 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.121652 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.121711 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:32.122092 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 08:45:32.122155 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":79,"threads_started":1}
I20260809 08:45:32.122607 22766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.117008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59628 (local address 127.31.63.66:41367)
0809 08:45:32.117800 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:32.117804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:32.117815 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:45:32.117865 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:32.117867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:32.117883 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:32.117969 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:32.118151 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.119314 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.120813 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.121643 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.122365 (+ 722us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.122443 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:32.122467 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 08:45:32.122502 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":102,"threads_started":1}
I20260809 08:45:32.123035 22645 tablet_service.cc:1467] Processing CreateTablet for tablet f8513f2ff47d4180aab1b469a8c88df0 (DEFAULT_TABLE table=test-table [id=08c9db7202914d23bbc3f7ba11af8a07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:32.123431 22645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8513f2ff47d4180aab1b469a8c88df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:32.123360 22450 tablet_service.cc:1467] Processing CreateTablet for tablet f8513f2ff47d4180aab1b469a8c88df0 (DEFAULT_TABLE table=test-table [id=08c9db7202914d23bbc3f7ba11af8a07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:32.123688 22450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8513f2ff47d4180aab1b469a8c88df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:32.125232 22768 tablet_bootstrap.cc:492] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Bootstrap starting.
I20260809 08:45:32.125955 22768 tablet_bootstrap.cc:654] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:32.126466 22768 log.cc:824] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:32.127261 22768 tablet_bootstrap.cc:492] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: No bootstrap required, opened a new log
I20260809 08:45:32.127228 22772 tablet_bootstrap.cc:492] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: Bootstrap starting.
I20260809 08:45:32.127420 22768 ts_tablet_manager.cc:1397] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:45:32.127918 22772 tablet_bootstrap.cc:654] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:32.128542 22772 log.cc:824] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:32.128897 22770 tablet_bootstrap.cc:492] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: Bootstrap starting.
I20260809 08:45:32.129339 22772 tablet_bootstrap.cc:492] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: No bootstrap required, opened a new log
I20260809 08:45:32.129478 22772 ts_tablet_manager.cc:1397] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:45:32.129604 22770 tablet_bootstrap.cc:654] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:32.130185 22770 log.cc:824] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:32.130061 22768 raft_consensus.cc:348] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:32.130260 22768 raft_consensus.cc:374] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:32.130299 22768 raft_consensus.cc:729] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac207fc59f2644c086ecb956acf15f19, State: Initialized, Role: FOLLOWER
I20260809 08:45:32.130466 22768 consensus_queue.cc:260] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:32.130841 22768 ts_tablet_manager.cc:1428] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:45:32.130978 22362 heartbeater.cc:503] Master 127.31.63.126:37669 was elected leader, sending a full tablet report...
I20260809 08:45:32.131055 22770 tablet_bootstrap.cc:492] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: No bootstrap required, opened a new log
I20260809 08:45:32.131119 22770 ts_tablet_manager.cc:1397] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:45:32.131635 22772 raft_consensus.cc:348] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:32.131829 22772 raft_consensus.cc:374] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:32.131916 22772 raft_consensus.cc:729] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df8657b7106472a98abc8cdeffdc431, State: Initialized, Role: FOLLOWER
I20260809 08:45:32.132084 22772 consensus_queue.cc:260] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:32.132423 22745 heartbeater.cc:503] Master 127.31.63.126:37669 was elected leader, sending a full tablet report...
I20260809 08:45:32.132436 22772 ts_tablet_manager.cc:1428] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:45:32.133051 22770 raft_consensus.cc:348] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:32.133255 22770 raft_consensus.cc:374] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:32.133314 22770 raft_consensus.cc:729] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b636bf8d5f9a458f926f8936b6ebe054, State: Initialized, Role: FOLLOWER
I20260809 08:45:32.133426 22770 consensus_queue.cc:260] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:32.133726 22770 ts_tablet_manager.cc:1428] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:45:32.133781 22537 heartbeater.cc:503] Master 127.31.63.126:37669 was elected leader, sending a full tablet report...
W20260809 08:45:32.167498 22539 tablet.cc:2369] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:45:32.256635 22363 tablet.cc:2369] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:45:32.333069 22746 tablet.cc:2369] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:45:32.473613 22775 raft_consensus.cc:482] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:32.473759 22775 raft_consensus.cc:504] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:32.474156 22775 leader_election.cc:288] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339), b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367)
I20260809 08:45:32.477607 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.474642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:41367 (local address 127.31.63.65:53575)
0809 08:45:32.474752 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:32.474763 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:32.474827 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:32.474967 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:32.474967 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:32.474969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:32.475082 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.475084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.475776 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.475777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.476698 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.476700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.477301 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.477302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.477376 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.477380 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:32.477447 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:45:32.477489 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":48,"threads_started":1}
I20260809 08:45:32.477593 22766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.474716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:53575 (local address 127.31.63.66:41367)
0809 08:45:32.474795 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:32.474797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:32.474810 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:45:32.474889 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:32.474890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:32.474904 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:32.474964 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:32.475176 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.475669 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.476804 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.477200 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.477291 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.477301 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:32.477455 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 08:45:32.477500 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 08:45:32.477898 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.474406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40339 (local address 127.31.63.65:33243)
0809 08:45:32.474543 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:32.474557 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:32.474615 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:32.474875 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:32.474877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:32.474879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:32.474973 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.474975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.476032 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.476033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.476804 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:32.476806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.477597 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.477598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:32.477704 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.477708 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:45:32.477770 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:45:32.477820 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260809 08:45:32.477903 22763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.474552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:33243 (local address 127.31.63.67:40339)
0809 08:45:32.474627 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:32.474629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:32.474644 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:45:32.474758 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:32.474759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:32.474772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:32.474843 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:32.475114 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.475911 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.476928 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.477435 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.477553 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.477584 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:32.477787 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 08:45:32.477839 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 08:45:32.477939 22480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8513f2ff47d4180aab1b469a8c88df0" candidate_uuid: "ac207fc59f2644c086ecb956acf15f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b636bf8d5f9a458f926f8936b6ebe054" is_pre_election: true
I20260809 08:45:32.478096 22480 raft_consensus.cc:2454] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac207fc59f2644c086ecb956acf15f19 in term 0.
I20260809 08:45:32.478370 22210 leader_election.cc:302] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac207fc59f2644c086ecb956acf15f19, b636bf8d5f9a458f926f8936b6ebe054; no voters:
I20260809 08:45:32.478400 22676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8513f2ff47d4180aab1b469a8c88df0" candidate_uuid: "ac207fc59f2644c086ecb956acf15f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df8657b7106472a98abc8cdeffdc431" is_pre_election: true
I20260809 08:45:32.478523 22775 raft_consensus.cc:2781] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:45:32.478578 22775 raft_consensus.cc:482] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:45:32.478581 22676 raft_consensus.cc:2454] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac207fc59f2644c086ecb956acf15f19 in term 0.
I20260809 08:45:32.478600 22775 raft_consensus.cc:3037] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:32.479408 22775 raft_consensus.cc:504] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:32.479555 22775 leader_election.cc:288] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 1 election: Requested vote from peers 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339), b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367)
I20260809 08:45:32.479789 22676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8513f2ff47d4180aab1b469a8c88df0" candidate_uuid: "ac207fc59f2644c086ecb956acf15f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df8657b7106472a98abc8cdeffdc431"
I20260809 08:45:32.479789 22480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8513f2ff47d4180aab1b469a8c88df0" candidate_uuid: "ac207fc59f2644c086ecb956acf15f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b636bf8d5f9a458f926f8936b6ebe054"
I20260809 08:45:32.479882 22480 raft_consensus.cc:3037] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:32.479882 22676 raft_consensus.cc:3037] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:32.480855 22676 raft_consensus.cc:2454] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac207fc59f2644c086ecb956acf15f19 in term 1.
I20260809 08:45:32.480855 22480 raft_consensus.cc:2454] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac207fc59f2644c086ecb956acf15f19 in term 1.
I20260809 08:45:32.481084 22210 leader_election.cc:302] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df8657b7106472a98abc8cdeffdc431, ac207fc59f2644c086ecb956acf15f19; no voters:
I20260809 08:45:32.481207 22775 raft_consensus.cc:2781] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:45:32.481395 22775 raft_consensus.cc:686] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 1 LEADER]: Becoming Leader. State: Replica: ac207fc59f2644c086ecb956acf15f19, State: Running, Role: LEADER
I20260809 08:45:32.481511 22775 consensus_queue.cc:237] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:32.482483 22102 catalog_manager.cc:5495] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac207fc59f2644c086ecb956acf15f19 (127.31.63.65). New cstate: current_term: 1 leader_uuid: "ac207fc59f2644c086ecb956acf15f19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } health_report { overall_health: UNKNOWN } } }
I20260809 08:45:32.562840 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.560156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.31.63.65:41003)
0809 08:45:32.560228 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:32.560230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:32.560239 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:45:32.560282 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:32.560282 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:32.560304 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:32.560384 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:45:32.560536 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.561066 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.561958 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:32.562175 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:32.562251 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:32.562530 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:45:32.562651 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:45:32.562723 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:32.562756 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 08:45:32.562775 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:45:32.564930 22676 raft_consensus.cc:1261] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431 [term 1 FOLLOWER]: Refusing update from remote peer ac207fc59f2644c086ecb956acf15f19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:32.564924 22480 raft_consensus.cc:1261] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054 [term 1 FOLLOWER]: Refusing update from remote peer ac207fc59f2644c086ecb956acf15f19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:32.565347 22775 consensus_queue.cc:1035] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:32.565503 22781 consensus_queue.cc:1035] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:32.568863 22788 mvcc.cc:205] Tried to move back new op lower bound from 7316541982982717440 to 7316541982645092352. Current Snapshot: MvccSnapshot[applied={T|T < 7316541982982717440}]
I20260809 08:45:32.569257 22790 mvcc.cc:205] Tried to move back new op lower bound from 7316541982982717440 to 7316541982645092352. Current Snapshot: MvccSnapshot[applied={T|T < 7316541982982717440}]
I20260809 08:45:32.570403 22102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:45:32.570506 22102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:45:32.571331 22102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Sending DeleteTablet for 3 replicas of tablet f8513f2ff47d4180aab1b469a8c88df0
I20260809 08:45:32.571714 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 22188
I20260809 08:45:32.571714 22274 tablet_service.cc:1514] Processing DeleteTablet for tablet f8513f2ff47d4180aab1b469a8c88df0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:32 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:58244
I20260809 08:45:32.571713 22450 tablet_service.cc:1514] Processing DeleteTablet for tablet f8513f2ff47d4180aab1b469a8c88df0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:32 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:59628
I20260809 08:45:32.579252 22645 tablet_service.cc:1514] Processing DeleteTablet for tablet f8513f2ff47d4180aab1b469a8c88df0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:32 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:43464
I20260809 08:45:32.579345 22797 tablet_replica.cc:330] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: stopping tablet replica
I20260809 08:45:32.579474 22797 raft_consensus.cc:2229] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:45:32.579495 22798 tablet_replica.cc:330] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: stopping tablet replica
I20260809 08:45:32.579579 22798 raft_consensus.cc:2229] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:45:32.579639 22797 raft_consensus.cc:2258] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:32.579663 22798 raft_consensus.cc:2258] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:32.580840 22797 ts_tablet_manager.cc:1905] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:32.583194 22797 ts_tablet_manager.cc:1918] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:32.583312 22797 log.cc:1196] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/f8513f2ff47d4180aab1b469a8c88df0
I20260809 08:45:32.583696 22797 ts_tablet_manager.cc:1939] T f8513f2ff47d4180aab1b469a8c88df0 P b636bf8d5f9a458f926f8936b6ebe054: Deleting consensus metadata
I20260809 08:45:32.584007 22798 ts_tablet_manager.cc:1905] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:32.584241 22083 catalog_manager.cc:4841] TS b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367): tablet f8513f2ff47d4180aab1b469a8c88df0 (table test-table [id=08c9db7202914d23bbc3f7ba11af8a07]) successfully deleted
I20260809 08:45:32.585419 22798 ts_tablet_manager.cc:1918] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:32.585480 22798 log.cc:1196] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/f8513f2ff47d4180aab1b469a8c88df0
I20260809 08:45:32.585752 22798 ts_tablet_manager.cc:1939] T f8513f2ff47d4180aab1b469a8c88df0 P 5df8657b7106472a98abc8cdeffdc431: Deleting consensus metadata
I20260809 08:45:32.586189 22083 catalog_manager.cc:4841] TS 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339): tablet f8513f2ff47d4180aab1b469a8c88df0 (table test-table [id=08c9db7202914d23bbc3f7ba11af8a07]) successfully deleted
W20260809 08:45:32.588631 22086 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:41003 (error 108)
I20260809 08:45:32.588828 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 22368
I20260809 08:45:32.592202 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:32.592045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:41003 (local address 127.0.0.1:58262)
0809 08:45:32.592104 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:32.592147 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:45:32.592331 22086 catalog_manager.cc:4584] TS ac207fc59f2644c086ecb956acf15f19 (127.31.63.65:41003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8513f2ff47d4180aab1b469a8c88df0: Network error: Client connection negotiation failed: client connection to 127.31.63.65:41003: connect: Connection refused (error 111)
I20260809 08:45:32.604503 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 22543
I20260809 08:45:32.612958 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 22064
I20260809 08:45:52.621187 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37669
--webserver_interface=127.31.63.126
--webserver_port=45731
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:37669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:45:52.707181 22803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:52.707392 22803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:52.707414 22803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:52.708571 22803 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 08:45:52.708616 22803 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 08:45:52.708629 22803 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 08:45:52.708642 22803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:45:52.708655 22803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:52.708667 22803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:52.708720 22803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:45:52.708737 22803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:45:52.710253 22803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:37669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:52.710518 22803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:52.710773 22803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:52.721051 22814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:52.721086 22812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:52.721168 22803 server_base.cc:1034] running on GCE node
W20260809 08:45:52.721122 22816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:52.721613 22803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:52.721966 22803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:52.723116 22803 hybrid_clock.cc:648] HybridClock initialized: now 1786265152723098 us; error 37 us; skew 500 ppm
Aug 09 08:45:52 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:52.729436 22803 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:45:52.729985 22803 webserver.cc:460] Webserver started at http://127.31.63.126:45731/ using document root <none> and password file <none>
I20260809 08:45:52.730244 22803 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:52.730322 22803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:52.731812 22803 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:45:52.732573 22823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:52.732774 22803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:45:52.732884 22803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "50373fff1d50469188bfd92cde0a8611"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "e044627a74c00856825476dedd90f232"
server_key_iv: "c4cc3c54aa323341b0e120685f128562"
server_key_version: "encryptionkey@0"
I20260809 08:45:52.733325 22803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:52.758148 22803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:52.760522 22803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:52.760687 22803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:52.765003 22803 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:37669
I20260809 08:45:52.765043 22895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:37669 every 8 connection(s)
I20260809 08:45:52.765468 22803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:45:52.766983 22896 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:45:52.767606 22896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Bootstrap starting.
I20260809 08:45:52.767910 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 22803
I20260809 08:45:52.768275 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:41003
--local_ip_for_outbound_sockets=127.31.63.65
--tserver_master_addrs=127.31.63.126:37669
--webserver_port=45639
--webserver_interface=127.31.63.65
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:52.777674 22896 log.cc:824] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:52.779513 22896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:45:52.779829 22896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Bootstrap complete.
I20260809 08:45:52.782325 22896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:45:52.782593 22896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50373fff1d50469188bfd92cde0a8611, State: Initialized, Role: FOLLOWER
I20260809 08:45:52.782778 22896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:45:52.782835 22896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:45:52.782893 22896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:45:52.782941 22896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:45:52.783927 22896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:45:52.784011 22896 leader_election.cc:302] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50373fff1d50469188bfd92cde0a8611; no voters:
I20260809 08:45:52.784192 22896 leader_election.cc:288] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:45:52.784252 22903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:45:52.784403 22903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 2 LEADER]: Becoming Leader. State: Replica: 50373fff1d50469188bfd92cde0a8611, State: Running, Role: LEADER
I20260809 08:45:52.784457 22896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:45:52.784580 22903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:45:52.785240 22905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50373fff1d50469188bfd92cde0a8611. Latest consensus state: current_term: 2 leader_uuid: "50373fff1d50469188bfd92cde0a8611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } } }
I20260809 08:45:52.785339 22905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:52.785521 22916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:45:52.785545 22904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "50373fff1d50469188bfd92cde0a8611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } } }
I20260809 08:45:52.785667 22904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: This master's current role is: LEADER
I20260809 08:45:52.786378 22916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:45:52.786532 22916 catalog_manager.cc:1259] Loaded cluster ID: 2cf68c810c9d4a4d8a2ebe605a6aca14
I20260809 08:45:52.786793 22916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:45:52.787961 22916 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:45:52.788120 22916 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Loaded TSK: 0
I20260809 08:45:52.798651 22916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Generated new TSK 1
I20260809 08:45:52.798797 22916 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:45:52.855064 22901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:52.855275 22901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:52.855299 22901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:52.856447 22901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:52.856489 22901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:52.856516 22901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:45:52.858008 22901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:41003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=45639
--tserver_master_addrs=127.31.63.126:37669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:52.858242 22901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:52.858485 22901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:52.864494 22945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:52.864517 22942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:52.864506 22941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:52.864946 22901 server_base.cc:1034] running on GCE node
I20260809 08:45:52.865164 22901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:52.865413 22901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:52.866580 22901 hybrid_clock.cc:648] HybridClock initialized: now 1786265152866577 us; error 44 us; skew 500 ppm
Aug 09 08:45:52 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:52.869328 22901 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:45:52.869832 22901 webserver.cc:460] Webserver started at http://127.31.63.65:45639/ using document root <none> and password file <none>
I20260809 08:45:52.870069 22901 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:52.870152 22901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:52.871618 22901 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:52.872385 22954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:52.872565 22901 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:52.872679 22901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "ac207fc59f2644c086ecb956acf15f19"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "03c3e72c85968851b7a34b5807949b7d"
server_key_iv: "89849320ff0c6074469874f54f0f0b13"
server_key_version: "encryptionkey@0"
I20260809 08:45:52.873057 22901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:52.904139 22901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:52.906472 22901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:52.906649 22901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:52.906916 22901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:52.907446 22965 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:45:52.909034 22901 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:45:52.909089 22901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:45:52.909116 22901 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:45:52.909879 22901 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:45:52.909929 22901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:52.909976 22965 tablet_bootstrap.cc:492] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Bootstrap starting.
I20260809 08:45:52.918452 22901 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:41003
I20260809 08:45:52.918547 23134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:41003 every 8 connection(s)
I20260809 08:45:52.918998 22901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:45:52.923115 22965 log.cc:824] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:52.924826 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 22901
I20260809 08:45:52.925179 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:41367
--local_ip_for_outbound_sockets=127.31.63.66
--tserver_master_addrs=127.31.63.126:37669
--webserver_port=36941
--webserver_interface=127.31.63.66
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:45:52.925565 22965 tablet_bootstrap.cc:492] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:45:52.925897 22965 tablet_bootstrap.cc:492] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Bootstrap complete.
I20260809 08:45:52.927327 22965 ts_tablet_manager.cc:1397] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.010s
Aug 09 08:45:52 dist-test-slave-gkw7 krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:52.928503 22965 raft_consensus.cc:348] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:52.928840 22965 raft_consensus.cc:729] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac207fc59f2644c086ecb956acf15f19, State: Initialized, Role: FOLLOWER
I20260809 08:45:52.929010 22965 consensus_queue.cc:260] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } }
I20260809 08:45:52.929343 22965 ts_tablet_manager.cc:1428] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:45:52.932556 23139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:52.919872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:53821 (local address 127.31.63.126:37669)
0809 08:45:52.920038 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:52.920048 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:52.920087 (+ 39us) server_negotiation.cc:407] Connection header received
0809 08:45:52.921087 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:52.921093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:52.921181 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:52.921304 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:52.922418 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:52.923394 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:52.924519 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:52.925185 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:52.925307 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:52.928614 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:52.928618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:52.928618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:52.928632 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:52.930521 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:52.931110 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:52.931112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:52.931113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:52.931284 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:52.931612 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:52.931613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:52.931614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:52.931853 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:52.931919 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:52.932303 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 08:45:52.932413 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260809 08:45:52.932704 23138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:52.919723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:53821)
0809 08:45:52.919939 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:52.920000 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:52.920837 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:52.921388 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:52.921393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:52.921919 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:52.922260 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:52.922270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:52.923548 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:52.923549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:52.924417 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:52.924420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:52.925509 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:52.925510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:52.925696 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:52.926369 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:52.926379 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:52.928490 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:52.930609 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:52.930618 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:52.930620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:52.931023 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:52.931371 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:52.931372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:52.931373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:52.931536 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:52.931925 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:52.931933 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:52.932119 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 08:45:52.932245 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 08:45:52.932451 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":116,"threads_started":1}
I20260809 08:45:52.933557 23135 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37669
I20260809 08:45:52.933672 23135 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:52.933897 23135 heartbeater.cc:511] Master 127.31.63.126:37669 requested a full tablet report, sending...
I20260809 08:45:52.934716 22850 ts_manager.cc:194] Registered new tserver with Master: ac207fc59f2644c086ecb956acf15f19 (127.31.63.65:41003)
I20260809 08:45:52.935811 22850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:53821
I20260809 08:45:52.936954 23135 heartbeater.cc:503] Master 127.31.63.126:37669 was elected leader, sending a full tablet report...
Aug 09 08:45:52 dist-test-slave-gkw7 krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265152, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.65@KRBTEST.COM
I20260809 08:45:52.943739 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:52.935329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:41003 (local address 127.0.0.1:57768)
0809 08:45:52.935681 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:52.935719 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:52.935801 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:52.936009 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:52.936014 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:52.936399 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:52.936450 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:52.936452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:52.937464 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:52.937466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:52.938087 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:52.938090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:52.938795 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:52.938796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:52.938859 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:52.939385 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:52.939390 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:52.940687 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:52.942659 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:52.942667 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:52.942668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:52.942863 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:52.943078 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:52.943078 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:52.943079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:52.943140 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:52.943443 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:52.943449 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:52.943496 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 08:45:52.943567 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:45:52.943666 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":102,"threads_started":1}
I20260809 08:45:52.943825 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:52.935421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.31.63.65:41003)
0809 08:45:52.935601 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:52.935606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:52.935806 (+ 200us) server_negotiation.cc:407] Connection header received
0809 08:45:52.935897 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:52.935900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:52.935923 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:52.935998 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:52.936543 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:52.937353 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:52.938187 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:52.938696 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:52.938794 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:52.940770 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:52.940778 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:52.940779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:52.940786 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:52.942571 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:52.942938 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:52.942942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:52.942942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:52.943012 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:52.943194 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:52.943194 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:52.943195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:52.943373 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:52.943455 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:52.943617 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 08:45:52.943727 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260809 08:45:52.944200 23032 tablet_service.cc:1514] Processing DeleteTablet for tablet f8513f2ff47d4180aab1b469a8c88df0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:32 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:57768
I20260809 08:45:52.944753 23145 tablet_replica.cc:330] stopping tablet replica
I20260809 08:45:52.944875 23145 raft_consensus.cc:2229] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:45:52.944957 23145 raft_consensus.cc:2258] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:52.945250 23145 ts_tablet_manager.cc:1905] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:52.947309 23145 ts_tablet_manager.cc:1918] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:52.947398 23145 log.cc:1196] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal/wals/f8513f2ff47d4180aab1b469a8c88df0
I20260809 08:45:52.947652 23145 ts_tablet_manager.cc:1939] T f8513f2ff47d4180aab1b469a8c88df0 P ac207fc59f2644c086ecb956acf15f19: Deleting consensus metadata
I20260809 08:45:52.948184 22829 catalog_manager.cc:4841] TS ac207fc59f2644c086ecb956acf15f19 (127.31.63.65:41003): tablet f8513f2ff47d4180aab1b469a8c88df0 (table test-table [id=08c9db7202914d23bbc3f7ba11af8a07]) successfully deleted
W20260809 08:45:53.015729 23141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:53.015971 23141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:53.016012 23141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:53.017161 23141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:53.017235 23141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:53.017282 23141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:45:53.018824 23141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:41367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=36941
--tserver_master_addrs=127.31.63.126:37669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:53.019104 23141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:53.019353 23141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:53.024688 23150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:53.024681 23154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:53.024681 23151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:53.024741 23141 server_base.cc:1034] running on GCE node
I20260809 08:45:53.025103 23141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:53.025398 23141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:53.026557 23141 hybrid_clock.cc:648] HybridClock initialized: now 1786265153026543 us; error 46 us; skew 500 ppm
Aug 09 08:45:53 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265153, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:53.029234 23141 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:45:53.029774 23141 webserver.cc:460] Webserver started at http://127.31.63.66:36941/ using document root <none> and password file <none>
I20260809 08:45:53.030032 23141 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:53.030118 23141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:53.031494 23141 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:53.032119 23162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:53.032305 23141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:53.032408 23141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "b636bf8d5f9a458f926f8936b6ebe054"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "12233195af3fcc5b329888c99b677622"
server_key_iv: "06b822c23c5654febc27538aecac2eca"
server_key_version: "encryptionkey@0"
I20260809 08:45:53.032747 23141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:53.042752 23141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:53.044812 23141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:53.044976 23141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:53.045215 23141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:53.045612 23141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:53.045666 23141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:53.045724 23141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:53.045748 23141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:53.052228 23141 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:41367
I20260809 08:45:53.052268 23325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:41367 every 8 connection(s)
I20260809 08:45:53.052634 23141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:45:53.053069 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 23141
I20260809 08:45:53.053396 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:40339
--local_ip_for_outbound_sockets=127.31.63.67
--tserver_master_addrs=127.31.63.126:37669
--webserver_port=38493
--webserver_interface=127.31.63.67
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:45:53 dist-test-slave-gkw7 krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265153, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:53.064738 23139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.053763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:38883 (local address 127.31.63.126:37669)
0809 08:45:53.053829 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:53.053831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:53.053871 (+ 40us) server_negotiation.cc:407] Connection header received
0809 08:45:53.054751 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:53.054752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:53.054762 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:53.054805 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:53.056150 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.056925 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.058772 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.059015 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.059094 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.061821 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:53.061823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:53.061824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:53.061837 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:53.063311 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:53.063742 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:53.063743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:53.063743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:53.063805 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:53.064062 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:53.064063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:53.064064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:53.064179 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:53.064266 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:53.064547 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 08:45:53.064640 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:45:53.064934 23328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.053471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:38883)
0809 08:45:53.053724 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:53.053782 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:53.054637 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:53.054899 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:53.054903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:53.055820 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:53.056042 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.056048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.057038 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.057040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.058678 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.058682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.059126 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.059127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.059317 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.059912 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:53.059925 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:53.061708 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:53.063414 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:53.063422 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:53.063424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:53.063659 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:53.063870 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:53.063871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:53.063872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:53.063991 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:53.064261 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:53.064270 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:53.064413 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 08:45:53.064512 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 08:45:53.064682 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":98,"threads_started":1}
I20260809 08:45:53.065451 23326 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37669
I20260809 08:45:53.065598 23326 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:53.065843 23326 heartbeater.cc:511] Master 127.31.63.126:37669 requested a full tablet report, sending...
I20260809 08:45:53.066295 22850 ts_manager.cc:194] Registered new tserver with Master: b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367)
I20260809 08:45:53.066908 22850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:38883
W20260809 08:45:53.142736 23329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:45:53.142939 23329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:45:53.142963 23329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:45:53.144039 23329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:45:53.144073 23329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:45:53.144099 23329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:45:53.145465 23329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:40339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=38493
--tserver_master_addrs=127.31.63.126:37669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:45:53.145722 23329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:45:53.145953 23329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:45:53.151090 23341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:45:53.151075 23340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:53.151129 23329 server_base.cc:1034] running on GCE node
W20260809 08:45:53.151062 23344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:45:53.151542 23329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:45:53.151804 23329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:45:53.152969 23329 hybrid_clock.cc:648] HybridClock initialized: now 1786265153152958 us; error 40 us; skew 500 ppm
Aug 09 08:45:53 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265153, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:45:53.155500 23329 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:45:53.155892 23329 webserver.cc:460] Webserver started at http://127.31.63.67:38493/ using document root <none> and password file <none>
I20260809 08:45:53.156070 23329 fs_manager.cc:359] Metadata directory not provided
I20260809 08:45:53.156123 23329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:45:53.157397 23329 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:45:53.158020 23355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:53.158200 23329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:53.158289 23329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "5df8657b7106472a98abc8cdeffdc431"
format_stamp: "Formatted at 2026-08-09 08:45:32 on dist-test-slave-gkw7"
server_key: "6343a7d2e72139f1d3edec0e9373660e"
server_key_iv: "2b515e23e65f5240599f8132d1b7a0d0"
server_key_version: "encryptionkey@0"
I20260809 08:45:53.158588 23329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:45:53.168292 23329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:45:53.170094 23329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:45:53.170203 23329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:45:53.170419 23329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:45:53.170722 23329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:45:53.170755 23329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:53.170795 23329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:45:53.170821 23329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:45:53.177038 23329 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:40339
I20260809 08:45:53.177106 23532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:40339 every 8 connection(s)
I20260809 08:45:53.177438 23329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:45:53.180570 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 23329
Aug 09 08:45:53 dist-test-slave-gkw7 krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265153, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:45:53.187796 23139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.178247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:35695 (local address 127.31.63.126:37669)
0809 08:45:53.178308 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:53.178310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:53.178526 (+ 216us) server_negotiation.cc:407] Connection header received
0809 08:45:53.179000 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:53.179001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:53.179012 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:53.179064 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:53.179965 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.180473 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.182039 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.182237 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.182318 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.184877 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:53.184879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:53.184879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:53.184888 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:53.186363 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:53.186779 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:53.186781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:53.186782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:53.186840 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:53.187122 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:53.187123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:53.187123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:53.187226 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:53.187310 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:53.187617 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 08:45:53.187700 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:45:53.187994 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.178239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:35695)
0809 08:45:53.178425 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:53.178459 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:53.178918 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:53.179136 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:53.179139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:53.179702 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:45:53.179889 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.179893 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.180566 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.180566 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.181959 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.181962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.182330 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.182332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.182470 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.183021 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:45:53.183030 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:45:53.184711 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:45:53.186449 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:53.186457 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:53.186459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:53.186690 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:53.186905 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:45:53.186906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:45:53.186907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:45:53.187056 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:45:53.187309 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:45:53.187316 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:45:53.187478 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 08:45:53.187588 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 08:45:53.187752 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260809 08:45:53.188076 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.181018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.31.63.126:37669)
0809 08:45:53.181133 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:53.181135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:53.181250 (+ 115us) server_negotiation.cc:407] Connection header received
0809 08:45:53.181308 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:53.181308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:53.181318 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:53.181372 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:53.182163 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.182670 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.183411 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.183653 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.183727 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.185318 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:53.185319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:53.185320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:53.185326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:53.186917 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:53.187285 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:53.187286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:53.187287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:53.187344 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:53.187547 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:53.187547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:53.187548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:53.187680 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:53.187866 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:53.187911 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 08:45:53.187999 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
I20260809 08:45:53.188647 23533 heartbeater.cc:347] Connected to a master server at 127.31.63.126:37669
I20260809 08:45:53.188746 23533 heartbeater.cc:464] Registering TS with master...
I20260809 08:45:53.188947 23533 heartbeater.cc:511] Master 127.31.63.126:37669 requested a full tablet report, sending...
I20260809 08:45:53.189311 22850 ts_manager.cc:194] Registered new tserver with Master: 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339)
I20260809 08:45:53.189733 22850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:35695
I20260809 08:45:53.199694 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:45:53.202221 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.200023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.31.63.126:37669)
0809 08:45:53.200071 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:53.200073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:53.200176 (+ 103us) server_negotiation.cc:407] Connection header received
0809 08:45:53.200230 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:53.200231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:53.200263 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:53.200321 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:45:53.200424 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.200875 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.201634 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.201834 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.201886 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.202022 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:45:53.202060 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 08:45:53.202178 (+ 118us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36426: token expired
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 08:45:53.202265 23538 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36426: token expired
W20260809 08:45:53.202571 22748 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.63.126:37669)
I20260809 08:45:53.208541 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.202838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36440 (local address 127.31.63.126:37669)
0809 08:45:53.202892 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:53.202894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:53.202903 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:45:53.202924 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:53.202924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:53.202932 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:53.202980 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:45:53.203475 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.203958 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.204751 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.204928 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.204997 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.206281 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:45:53.206282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:45:53.206283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:45:53.206289 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:45:53.207457 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:53.207796 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:53.207798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:53.207798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:53.207853 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:45:53.208047 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:45:53.208048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:45:53.208049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:45:53.208151 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:45:53.208249 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:53.208373 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 08:45:53.208448 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:45:53.209870 22850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:45:53.211437 22850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:45:53.213852 23032 tablet_service.cc:1467] Processing CreateTablet for tablet f5ff45f8a3af4a3a9adac38ff1aea5bf (DEFAULT_TABLE table=test-table [id=cb9f0d95277f499498e27258b734b481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:53.214066 23032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5ff45f8a3af4a3a9adac38ff1aea5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:53.215701 22965 tablet_bootstrap.cc:492] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: Bootstrap starting.
I20260809 08:45:53.216049 22965 tablet_bootstrap.cc:654] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:53.216676 22965 tablet_bootstrap.cc:492] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: No bootstrap required, opened a new log
I20260809 08:45:53.216723 22965 ts_tablet_manager.cc:1397] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 08:45:53.217067 22965 raft_consensus.cc:348] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:45:53.217161 22965 raft_consensus.cc:374] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:53.217217 22965 raft_consensus.cc:729] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac207fc59f2644c086ecb956acf15f19, State: Initialized, Role: FOLLOWER
I20260809 08:45:53.217262 22965 consensus_queue.cc:260] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:45:53.217360 22965 ts_tablet_manager.cc:1428] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 08:45:53.218251 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.213692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:41367 (local address 127.0.0.1:33612)
0809 08:45:53.213743 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:53.213755 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:53.213794 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:53.214494 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:53.214494 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:53.214496 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:53.214555 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.214557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.215645 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.215646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.216889 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.216892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.217944 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.217945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.218022 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.218056 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 08:45:53.218124 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 08:45:53.218172 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":60}
I20260809 08:45:53.218272 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.214019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33612 (local address 127.31.63.66:41367)
0809 08:45:53.214235 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:53.214240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:53.214253 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:45:53.214342 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:53.214345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:53.214365 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:53.214713 (+ 348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:53.214816 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.215553 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.216981 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.217796 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.217872 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.217931 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:53.218153 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 08:45:53.218202 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260809 08:45:53.218611 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.213812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40339 (local address 127.0.0.1:48196)
0809 08:45:53.213974 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:53.213985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:53.214036 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:53.214499 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:53.214500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:53.214503 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:53.214549 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.214551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.215855 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.215857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.216954 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.216957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.218372 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.218373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.218456 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.218459 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:45:53.218514 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:45:53.218553 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260809 08:45:53.218674 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.213917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.31.63.67:40339)
0809 08:45:53.214325 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:53.214330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:53.214339 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:45:53.214375 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:53.214377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:53.214395 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:53.214473 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:53.214645 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.215751 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.217035 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.217796 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.217872 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.217931 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:53.218544 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 08:45:53.218591 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":94,"threads_started":1}
I20260809 08:45:53.219112 23232 tablet_service.cc:1467] Processing CreateTablet for tablet f5ff45f8a3af4a3a9adac38ff1aea5bf (DEFAULT_TABLE table=test-table [id=cb9f0d95277f499498e27258b734b481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:53.219471 23232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5ff45f8a3af4a3a9adac38ff1aea5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:53.219448 23435 tablet_service.cc:1467] Processing CreateTablet for tablet f5ff45f8a3af4a3a9adac38ff1aea5bf (DEFAULT_TABLE table=test-table [id=cb9f0d95277f499498e27258b734b481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:45:53.219774 23435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5ff45f8a3af4a3a9adac38ff1aea5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:45:53.223296 23546 tablet_bootstrap.cc:492] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: Bootstrap starting.
I20260809 08:45:53.223886 23546 tablet_bootstrap.cc:654] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:53.224502 23546 log.cc:824] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:53.224864 23547 tablet_bootstrap.cc:492] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: Bootstrap starting.
I20260809 08:45:53.225472 23547 tablet_bootstrap.cc:654] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: Neither blocks nor log segments found. Creating new log.
I20260809 08:45:53.225878 23546 tablet_bootstrap.cc:492] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: No bootstrap required, opened a new log
I20260809 08:45:53.225962 23546 ts_tablet_manager.cc:1397] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:45:53.226097 23547 log.cc:824] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: Log is configured to *not* fsync() on all Append() calls
I20260809 08:45:53.226797 23547 tablet_bootstrap.cc:492] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: No bootstrap required, opened a new log
I20260809 08:45:53.226852 23547 ts_tablet_manager.cc:1397] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:45:53.227614 23546 raft_consensus.cc:348] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:45:53.227771 23546 raft_consensus.cc:374] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:53.227923 23546 raft_consensus.cc:729] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df8657b7106472a98abc8cdeffdc431, State: Initialized, Role: FOLLOWER
I20260809 08:45:53.228153 23546 consensus_queue.cc:260] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:45:53.228477 23546 ts_tablet_manager.cc:1428] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:45:53.228499 23533 heartbeater.cc:503] Master 127.31.63.126:37669 was elected leader, sending a full tablet report...
I20260809 08:45:53.228655 23547 raft_consensus.cc:348] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:45:53.228866 23547 raft_consensus.cc:374] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:45:53.228919 23547 raft_consensus.cc:729] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b636bf8d5f9a458f926f8936b6ebe054, State: Initialized, Role: FOLLOWER
I20260809 08:45:53.229030 23547 consensus_queue.cc:260] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:45:53.229341 23547 ts_tablet_manager.cc:1428] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:45:53.229365 23326 heartbeater.cc:503] Master 127.31.63.126:37669 was elected leader, sending a full tablet report...
I20260809 08:45:53.246608 23142 raft_consensus.cc:482] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:45:53.246738 23142 raft_consensus.cc:504] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:45:53.247017 23142 leader_election.cc:288] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367), 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339)
I20260809 08:45:53.251243 23138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.247213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:41367 (local address 127.31.63.65:39197)
0809 08:45:53.247272 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:53.247285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:53.247341 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:53.247599 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:53.247600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:53.247602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:53.247710 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.247713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.248670 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.248672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.250023 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.250026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.250934 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.250935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.251017 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.251047 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 08:45:53.251118 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:45:53.251165 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":7}
I20260809 08:45:53.251242 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.247278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:39197 (local address 127.31.63.66:41367)
0809 08:45:53.247346 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:53.247348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:53.247363 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:45:53.247435 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:53.247436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:53.247448 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:53.247518 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:53.247792 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.248567 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.250158 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.250854 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.250935 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.250945 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:53.251126 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 08:45:53.251168 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:45:53.251243 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.247433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:34389 (local address 127.31.63.67:40339)
0809 08:45:53.247491 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:53.247493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:53.247584 (+ 91us) server_negotiation.cc:407] Connection header received
0809 08:45:53.247661 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:53.247662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:53.247669 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:53.247712 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:45:53.247868 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.248567 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.250134 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.250809 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.250858 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.250882 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:53.251131 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 08:45:53.251167 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:45:53.251278 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.247340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40339 (local address 127.31.63.65:34389)
0809 08:45:53.247527 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:53.247539 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:45:53.247599 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:45:53.247731 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:45:53.247732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:45:53.247733 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:45:53.247798 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.247799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.248670 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.248672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.250023 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:45:53.250026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.250968 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.250969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:45:53.251023 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.251044 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 08:45:53.251111 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:45:53.251157 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":96,"thread_start_us":114,"threads_started":1}
I20260809 08:45:53.251662 23260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ff45f8a3af4a3a9adac38ff1aea5bf" candidate_uuid: "ac207fc59f2644c086ecb956acf15f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b636bf8d5f9a458f926f8936b6ebe054" is_pre_election: true
I20260809 08:45:53.251679 23460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ff45f8a3af4a3a9adac38ff1aea5bf" candidate_uuid: "ac207fc59f2644c086ecb956acf15f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df8657b7106472a98abc8cdeffdc431" is_pre_election: true
I20260809 08:45:53.251816 23260 raft_consensus.cc:2454] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac207fc59f2644c086ecb956acf15f19 in term 0.
I20260809 08:45:53.251820 23460 raft_consensus.cc:2454] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac207fc59f2644c086ecb956acf15f19 in term 0.
I20260809 08:45:53.252040 22955 leader_election.cc:302] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac207fc59f2644c086ecb956acf15f19, b636bf8d5f9a458f926f8936b6ebe054; no voters:
I20260809 08:45:53.252180 23142 raft_consensus.cc:2781] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:45:53.252244 23142 raft_consensus.cc:482] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:45:53.252274 23142 raft_consensus.cc:3037] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:53.252976 23142 raft_consensus.cc:504] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:45:53.253120 23142 leader_election.cc:288] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 1 election: Requested vote from peers b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367), 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339)
I20260809 08:45:53.253310 23260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ff45f8a3af4a3a9adac38ff1aea5bf" candidate_uuid: "ac207fc59f2644c086ecb956acf15f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b636bf8d5f9a458f926f8936b6ebe054"
I20260809 08:45:53.253337 23460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5ff45f8a3af4a3a9adac38ff1aea5bf" candidate_uuid: "ac207fc59f2644c086ecb956acf15f19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df8657b7106472a98abc8cdeffdc431"
I20260809 08:45:53.253405 23460 raft_consensus.cc:3037] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:53.253412 23260 raft_consensus.cc:3037] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:45:53.254142 23460 raft_consensus.cc:2454] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac207fc59f2644c086ecb956acf15f19 in term 1.
I20260809 08:45:53.254267 23260 raft_consensus.cc:2454] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac207fc59f2644c086ecb956acf15f19 in term 1.
I20260809 08:45:53.254307 22955 leader_election.cc:302] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df8657b7106472a98abc8cdeffdc431, ac207fc59f2644c086ecb956acf15f19; no voters:
I20260809 08:45:53.254400 23142 raft_consensus.cc:2781] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:45:53.254608 23142 raft_consensus.cc:686] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 LEADER]: Becoming Leader. State: Replica: ac207fc59f2644c086ecb956acf15f19, State: Running, Role: LEADER
I20260809 08:45:53.254684 23142 consensus_queue.cc:237] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:45:53.255441 22850 catalog_manager.cc:5495] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac207fc59f2644c086ecb956acf15f19 (127.31.63.65). New cstate: current_term: 1 leader_uuid: "ac207fc59f2644c086ecb956acf15f19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } health_report { overall_health: HEALTHY } } }
I20260809 08:45:53.265277 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.263152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57776 (local address 127.31.63.65:41003)
0809 08:45:53.263207 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:45:53.263209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:45:53.263238 (+ 29us) server_negotiation.cc:407] Connection header received
0809 08:45:53.263273 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:45:53.263273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:45:53.263289 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:45:53.263340 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:45:53.263438 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.263903 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.264595 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:45:53.264794 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:45:53.264842 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:45:53.264962 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:45:53.265051 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:45:53.265104 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:45:53.265167 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 08:45:53.265186 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:45:53.266779 23260 raft_consensus.cc:1261] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054 [term 1 FOLLOWER]: Refusing update from remote peer ac207fc59f2644c086ecb956acf15f19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:53.266795 23460 raft_consensus.cc:1261] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431 [term 1 FOLLOWER]: Refusing update from remote peer ac207fc59f2644c086ecb956acf15f19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:45:53.267117 23554 consensus_queue.cc:1035] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:53.267231 23142 consensus_queue.cc:1035] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:45:53.270023 23559 mvcc.cc:205] Tried to move back new op lower bound from 7316542067778473984 to 7316542067731927040. Current Snapshot: MvccSnapshot[applied={T|T < 7316542067778473984}]
I20260809 08:45:53.270402 23560 mvcc.cc:205] Tried to move back new op lower bound from 7316542067778473984 to 7316542067731927040. Current Snapshot: MvccSnapshot[applied={T|T < 7316542067778473984}]
I20260809 08:45:53.272203 22849 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:45:53.272301 22849 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:45:53.273154 22849 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Sending DeleteTablet for 3 replicas of tablet f5ff45f8a3af4a3a9adac38ff1aea5bf
I20260809 08:45:53.273468 23032 tablet_service.cc:1514] Processing DeleteTablet for tablet f5ff45f8a3af4a3a9adac38ff1aea5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:53 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:57768
I20260809 08:45:53.273586 23232 tablet_service.cc:1514] Processing DeleteTablet for tablet f5ff45f8a3af4a3a9adac38ff1aea5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:53 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:33612
I20260809 08:45:53.273821 23435 tablet_service.cc:1514] Processing DeleteTablet for tablet f5ff45f8a3af4a3a9adac38ff1aea5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:45:53 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:48196
I20260809 08:45:53.273864 23571 tablet_replica.cc:330] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: stopping tablet replica
I20260809 08:45:53.274039 23571 raft_consensus.cc:2229] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:45:53.274045 23572 tablet_replica.cc:330] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: stopping tablet replica
I20260809 08:45:53.274063 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 22901
I20260809 08:45:53.274231 23571 raft_consensus.cc:2258] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:53.278838 23572 raft_consensus.cc:2229] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:45:53.279024 23572 raft_consensus.cc:2258] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:45:53.279585 23572 ts_tablet_manager.cc:1905] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:53.281591 23572 ts_tablet_manager.cc:1918] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:53.281692 23572 log.cc:1196] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/f5ff45f8a3af4a3a9adac38ff1aea5bf
I20260809 08:45:53.282124 23572 ts_tablet_manager.cc:1939] T f5ff45f8a3af4a3a9adac38ff1aea5bf P 5df8657b7106472a98abc8cdeffdc431: Deleting consensus metadata
I20260809 08:45:53.283133 22826 catalog_manager.cc:4841] TS 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339): tablet f5ff45f8a3af4a3a9adac38ff1aea5bf (table test-table [id=cb9f0d95277f499498e27258b734b481]) successfully deleted
I20260809 08:45:53.284209 23571 ts_tablet_manager.cc:1905] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:45:53.286598 23571 ts_tablet_manager.cc:1918] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:45:53.286698 23571 log.cc:1196] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/f5ff45f8a3af4a3a9adac38ff1aea5bf
I20260809 08:45:53.287092 23571 ts_tablet_manager.cc:1939] T f5ff45f8a3af4a3a9adac38ff1aea5bf P b636bf8d5f9a458f926f8936b6ebe054: Deleting consensus metadata
I20260809 08:45:53.287633 22826 catalog_manager.cc:4841] TS b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367): tablet f5ff45f8a3af4a3a9adac38ff1aea5bf (table test-table [id=cb9f0d95277f499498e27258b734b481]) successfully deleted
I20260809 08:45:53.296707 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 23141
W20260809 08:45:53.296738 22829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:41003 (error 108)
I20260809 08:45:53.306107 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:45:53.305940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:41003 (local address 127.0.0.1:57786)
0809 08:45:53.306001 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 08:45:53.306056 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 08:45:53.306409 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 23329
W20260809 08:45:53.306298 22829 catalog_manager.cc:4584] TS ac207fc59f2644c086ecb956acf15f19 (127.31.63.65:41003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f5ff45f8a3af4a3a9adac38ff1aea5bf: Network error: Client connection negotiation failed: client connection to 127.31.63.65:41003: connect: Connection refused (error 111)
I20260809 08:45:53.313479 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 22803
I20260809 08:46:13.323460 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37669
--webserver_interface=127.31.63.126
--webserver_port=45731
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:37669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:13.414587 23576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:13.414805 23576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:13.414829 23576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:13.416612 23576 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 08:46:13.416719 23576 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 08:46:13.416754 23576 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 08:46:13.416770 23576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:13.416793 23576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:13.416813 23576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:13.416841 23576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:13.416887 23576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:13.418931 23576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:37669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:37669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:13.419150 23576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:13.419405 23576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:13.424561 23586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:13.424690 23576 server_base.cc:1034] running on GCE node
W20260809 08:46:13.424604 23583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:13.424540 23584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:13.425019 23576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:13.425359 23576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:13.426527 23576 hybrid_clock.cc:648] HybridClock initialized: now 1786265173426499 us; error 45 us; skew 500 ppm
Aug 09 08:46:13 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265173, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:13.433430 23576 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:46:13.433976 23576 webserver.cc:460] Webserver started at http://127.31.63.126:45731/ using document root <none> and password file <none>
I20260809 08:46:13.434221 23576 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:13.434295 23576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:13.435791 23576 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:46:13.436503 23600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:13.436734 23576 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:13.436851 23576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "50373fff1d50469188bfd92cde0a8611"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "e044627a74c00856825476dedd90f232"
server_key_iv: "c4cc3c54aa323341b0e120685f128562"
server_key_version: "encryptionkey@0"
I20260809 08:46:13.437232 23576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:13.447615 23576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:13.449673 23576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:13.449846 23576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:13.454051 23576 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:37669
I20260809 08:46:13.454082 23673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:37669 every 8 connection(s)
I20260809 08:46:13.454484 23576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:13.456035 23675 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:46:13.456746 23675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Bootstrap starting.
I20260809 08:46:13.459965 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 23576
I20260809 08:46:13.460304 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:41003
--local_ip_for_outbound_sockets=127.31.63.65
--tserver_master_addrs=127.31.63.126:37669
--webserver_port=45639
--webserver_interface=127.31.63.65
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:46:13.466841 23675 log.cc:824] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:13.469158 23675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:46:13.469491 23675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Bootstrap complete.
I20260809 08:46:13.471931 23675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:46:13.472185 23675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50373fff1d50469188bfd92cde0a8611, State: Initialized, Role: FOLLOWER
I20260809 08:46:13.472378 23675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:46:13.472458 23675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:13.472513 23675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:13.472561 23675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 2 FOLLOWER]: Advancing to term 3
I20260809 08:46:13.473623 23675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:46:13.473748 23675 leader_election.cc:302] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50373fff1d50469188bfd92cde0a8611; no voters:
I20260809 08:46:13.473963 23675 leader_election.cc:288] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 08:46:13.474000 23679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 08:46:13.474124 23679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [term 3 LEADER]: Becoming Leader. State: Replica: 50373fff1d50469188bfd92cde0a8611, State: Running, Role: LEADER
I20260809 08:46:13.474261 23675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:13.474305 23679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } }
I20260809 08:46:13.474761 23682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50373fff1d50469188bfd92cde0a8611. Latest consensus state: current_term: 3 leader_uuid: "50373fff1d50469188bfd92cde0a8611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } } }
I20260809 08:46:13.474857 23682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:13.474779 23680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "50373fff1d50469188bfd92cde0a8611" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50373fff1d50469188bfd92cde0a8611" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 37669 } } }
I20260809 08:46:13.475055 23680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:13.475543 23691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:13.476469 23691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:13.476574 23691 catalog_manager.cc:1259] Loaded cluster ID: 2cf68c810c9d4a4d8a2ebe605a6aca14
I20260809 08:46:13.476606 23691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:13.478020 23691 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:13.478201 23691 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Loaded TSK: 0
I20260809 08:46:13.478247 23691 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 50373fff1d50469188bfd92cde0a8611: Loaded TSK: 1
F20260809 08:46:13.478505 23691 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3bb21709ad google::LogMessage::Fail() at ??:0
@ 0x7f3bb217388a google::LogMessage::SendToLog() at ??:0
@ 0x7f3bb21704de google::LogMessage::Flush() at ??:0
@ 0x7f3bb2171a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3bb48dd104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3bb48dad41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3bba0ca1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3bba0ca48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3bba0ca594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f3bba090c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f3bba09aad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3bb403e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3bb4033954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3bb4e976db start_thread at ??:0
@ 0x7f3bb0c6e61f clone at ??:0
W20260809 08:46:13.568154 23677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:13.568367 23677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:13.568390 23677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:13.569590 23677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:13.569636 23677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:13.569664 23677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:46:13.571178 23677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:41003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=45639
--tserver_master_addrs=127.31.63.126:37669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:13.571450 23677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:13.571699 23677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:13.577003 23731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:13.577107 23677 server_base.cc:1034] running on GCE node
W20260809 08:46:13.577214 23730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:13.577287 23733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:13.577608 23677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:13.577914 23677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:13.579079 23677 hybrid_clock.cc:648] HybridClock initialized: now 1786265173579066 us; error 33 us; skew 500 ppm
Aug 09 08:46:13 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265173, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:13.581801 23677 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:46:13.582330 23677 webserver.cc:460] Webserver started at http://127.31.63.65:45639/ using document root <none> and password file <none>
I20260809 08:46:13.582572 23677 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:13.582645 23677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:13.584035 23677 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:46:13.584792 23744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:13.584975 23677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:13.585043 23677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "ac207fc59f2644c086ecb956acf15f19"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "03c3e72c85968851b7a34b5807949b7d"
server_key_iv: "89849320ff0c6074469874f54f0f0b13"
server_key_version: "encryptionkey@0"
I20260809 08:46:13.585426 23677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:13.608377 23677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:13.610714 23677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:13.610867 23677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:13.611155 23677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:13.611667 23753 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:46:13.613754 23677 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:46:13.613822 23677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:46:13.613873 23677 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:46:13.614549 23677 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:46:13.614611 23677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 08:46:13.614693 23753 tablet_bootstrap.cc:492] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: Bootstrap starting.
I20260809 08:46:13.622696 23677 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:41003
I20260809 08:46:13.623090 23677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:46:13.623692 23926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:41003 every 8 connection(s)
I20260809 08:46:13.626845 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 23677
I20260809 08:46:13.627170 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:41367
--local_ip_for_outbound_sockets=127.31.63.66
--tserver_master_addrs=127.31.63.126:37669
--webserver_port=36941
--webserver_interface=127.31.63.66
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:46:13.627794 23753 log.cc:824] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:13.630323 23753 tablet_bootstrap.cc:492] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:46:13.630623 23753 tablet_bootstrap.cc:492] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: Bootstrap complete.
I20260809 08:46:13.631675 23753 ts_tablet_manager.cc:1397] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.009s
I20260809 08:46:13.632809 23753 raft_consensus.cc:348] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:46:13.633064 23753 raft_consensus.cc:729] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac207fc59f2644c086ecb956acf15f19, State: Initialized, Role: FOLLOWER
I20260809 08:46:13.633275 23753 consensus_queue.cc:260] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:46:13.633621 23753 ts_tablet_manager.cc:1428] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260809 08:46:13.704049 23931 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:46:13.623908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:46531)
0809 08:46:13.624109 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:13.624147 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:13.624606 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:13.703762 (+ 79156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":111,"threads_started":1}
W20260809 08:46:13.704330 23927 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:37669 (0 consecutive failures): Network error: Failed to ping master at 127.31.63.126:37669: Client connection negotiation failed: client connection to 127.31.63.126:37669: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 08:46:13.704756 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:13.704626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:39949)
0809 08:46:13.704685 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:13.704734 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:46:13.705160 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:13.705036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:38841)
0809 08:46:13.705120 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:13.705143 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 08:46:13.705281 23927 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:46:13.726866 23933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:13.727101 23933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:13.727144 23933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:13.728315 23933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:13.728386 23933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:13.728432 23933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:46:13.730096 23933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:41367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=36941
--tserver_master_addrs=127.31.63.126:37669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:13.730388 23933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:13.730634 23933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:13.735980 23942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:13.735985 23944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:13.735975 23947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:13.736297 23933 server_base.cc:1034] running on GCE node
I20260809 08:46:13.736523 23933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:13.736810 23933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:13.738008 23933 hybrid_clock.cc:648] HybridClock initialized: now 1786265173737978 us; error 39 us; skew 500 ppm
Aug 09 08:46:13 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265173, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:13.740600 23933 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:46:13.741061 23933 webserver.cc:460] Webserver started at http://127.31.63.66:36941/ using document root <none> and password file <none>
I20260809 08:46:13.741340 23933 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:13.741413 23933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:13.742832 23933 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:13.743533 23957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:13.743734 23933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:13.743842 23933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "b636bf8d5f9a458f926f8936b6ebe054"
format_stamp: "Formatted at 2026-08-09 08:45:31 on dist-test-slave-gkw7"
server_key: "12233195af3fcc5b329888c99b677622"
server_key_iv: "06b822c23c5654febc27538aecac2eca"
server_key_version: "encryptionkey@0"
I20260809 08:46:13.744234 23933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:13.757890 23933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:13.759719 23933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:13.759882 23933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:13.760136 23933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:13.760447 23933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:13.760500 23933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:13.760541 23933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:13.760571 23933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:13.766852 23933 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:41367
I20260809 08:46:13.766904 24122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:41367 every 8 connection(s)
I20260809 08:46:13.767289 23933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:46:13.768349 24126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:13.767974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:51963)
0809 08:46:13.768131 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:13.768204 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
W20260809 08:46:13.768775 24124 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:37669 (0 consecutive failures): Network error: Failed to ping master at 127.31.63.126:37669: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
I20260809 08:46:13.769091 24126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:13.768995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:36477)
0809 08:46:13.769044 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:13.769072 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 08:46:13.769366 24126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:13.769317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:42415)
0809 08:46:13.769337 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:13.769354 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 08:46:13.769491 24124 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:46:13.774462 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 23933
I20260809 08:46:13.774736 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:40339
--local_ip_for_outbound_sockets=127.31.63.67
--tserver_master_addrs=127.31.63.126:37669
--webserver_port=38493
--webserver_interface=127.31.63.67
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:13.857802 24127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:13.858001 24127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:13.858024 24127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:13.859200 24127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:13.859249 24127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:13.859277 24127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:46:13.860760 24127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:40339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=38493
--tserver_master_addrs=127.31.63.126:37669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:13.861007 24127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:13.861260 24127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:13.866488 24137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:13.866480 24135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:13.866480 24139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:13.866670 24127 server_base.cc:1034] running on GCE node
I20260809 08:46:13.867024 24127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:13.867338 24127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:13.868503 24127 hybrid_clock.cc:648] HybridClock initialized: now 1786265173868491 us; error 45 us; skew 500 ppm
Aug 09 08:46:13 dist-test-slave-gkw7 krb5kdc[22047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265173, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:13.870965 24127 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:46:13.871481 24127 webserver.cc:460] Webserver started at http://127.31.63.67:38493/ using document root <none> and password file <none>
I20260809 08:46:13.871726 24127 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:13.871798 24127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:13.873150 24127 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:13.873863 24147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:13.874048 24127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:13.874131 24127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "5df8657b7106472a98abc8cdeffdc431"
format_stamp: "Formatted at 2026-08-09 08:45:32 on dist-test-slave-gkw7"
server_key: "6343a7d2e72139f1d3edec0e9373660e"
server_key_iv: "2b515e23e65f5240599f8132d1b7a0d0"
server_key_version: "encryptionkey@0"
I20260809 08:46:13.874475 24127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:13.886650 24127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:13.888518 24127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:13.888655 24127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:13.888907 24127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:13.889225 24127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:13.889285 24127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:13.889326 24127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:13.889353 24127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:13.895777 24127 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:40339
I20260809 08:46:13.895828 24333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:40339 every 8 connection(s)
I20260809 08:46:13.896178 24127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:46:13.897238 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:13.896814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:40693)
0809 08:46:13.896973 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:13.897088 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
W20260809 08:46:13.897732 24334 heartbeater.cc:644] Failed to heartbeat to 127.31.63.126:37669 (0 consecutive failures): Network error: Failed to ping master at 127.31.63.126:37669: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
I20260809 08:46:13.898088 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:13.897987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:38327)
0809 08:46:13.898040 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:13.898068 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 08:46:13.898463 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:13.898369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:60699)
0809 08:46:13.898409 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:13.898448 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 08:46:13.898584 24334 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:46:13.900681 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 24127
I20260809 08:46:14.705931 24338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:14.705679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:54421)
0809 08:46:14.705824 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:14.705865 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260809 08:46:14.770136 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:14.769882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:35223)
0809 08:46:14.770033 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:14.770080 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260809 08:46:14.899168 24340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:14.898931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:41645)
0809 08:46:14.899074 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:14.899115 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":61,"threads_started":1}
I20260809 08:46:15.268342 24341 raft_consensus.cc:482] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:15.268473 24341 raft_consensus.cc:504] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:46:15.268787 24341 leader_election.cc:288] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367), 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339)
Aug 09 08:46:15 dist-test-slave-gkw7 krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265173, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.67@KRBTEST.COM
Aug 09 08:46:15 dist-test-slave-gkw7 krb5kdc[22047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265173, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.66@KRBTEST.COM
I20260809 08:46:15.281896 24342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:15.268990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:41367 (local address 127.31.63.65:35875)
0809 08:46:15.269105 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:15.269117 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:15.269182 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:15.270256 (+ 1074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:15.270259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:15.270707 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:15.270896 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:15.270902 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:15.272045 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:15.272046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:15.273167 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:15.273171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:15.274163 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:15.274165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:15.274343 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:15.275135 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:15.275141 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:15.277427 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:15.279794 (+ 2367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:15.279832 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:15.279834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:15.280247 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:15.280588 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:15.280589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:15.280589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:15.280735 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:15.281275 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:15.281276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:15.281428 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 08:46:15.281555 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 08:46:15.281765 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":59,"thread_start_us":67,"threads_started":1}
I20260809 08:46:15.282029 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:15.269325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:33383 (local address 127.31.63.67:40339)
0809 08:46:15.269464 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:15.269474 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:15.269512 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:46:15.270030 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:15.270032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:15.270112 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:15.270284 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:15.271029 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:15.271909 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:15.273282 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:15.274057 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:15.274177 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:15.277320 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:15.277326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:15.277326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:15.277340 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:15.279808 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:15.280356 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:15.280359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:15.280360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:15.280522 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:15.280819 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:15.280820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:15.280821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:15.281165 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:15.281249 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:15.281753 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 08:46:15.281903 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
I20260809 08:46:15.282037 24343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:15.269174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40339 (local address 127.31.63.65:33383)
0809 08:46:15.269328 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:15.269338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:15.269386 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:15.270261 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:15.270262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:15.270708 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:15.270896 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:15.270902 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:15.272020 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:15.272021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:15.273168 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:15.273170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:15.274277 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:15.274278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:15.274362 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:15.275111 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:15.275123 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:15.277191 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:15.279907 (+ 2716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:15.279908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:15.279909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:15.280262 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:15.280618 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:15.280619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:15.280620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:15.280738 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:15.281246 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:15.281254 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:15.281429 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 08:46:15.281576 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 08:46:15.281873 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":164,"thread_start_us":81,"threads_started":1}
I20260809 08:46:15.283015 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:15.269125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:35875 (local address 127.31.63.66:41367)
0809 08:46:15.269339 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:15.269355 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:15.269401 (+ 46us) server_negotiation.cc:407] Connection header received
0809 08:46:15.269990 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:15.269993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:15.270040 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:15.270197 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:15.271028 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:15.271909 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:15.273264 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:15.274050 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:15.274183 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:15.277591 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:15.277641 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:15.277642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:15.277658 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:15.279649 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:15.280351 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:15.280354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:15.280355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:15.280490 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:15.280839 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:15.280840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:15.280841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:15.281164 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:15.282661 (+ 1497us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:15.282744 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 08:46:15.282895 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":122,"threads_started":1}
W20260809 08:46:15.283601 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
W20260809 08:46:15.283679 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
I20260809 08:46:15.283697 23745 leader_election.cc:302] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac207fc59f2644c086ecb956acf15f19; no voters: 5df8657b7106472a98abc8cdeffdc431, b636bf8d5f9a458f926f8936b6ebe054
I20260809 08:46:15.283823 24341 raft_consensus.cc:2726] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:46:15.706529 24343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:15.706396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:51565)
0809 08:46:15.706452 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:15.706488 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 08:46:15.770964 24347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:15.770717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:44431)
0809 08:46:15.770876 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:15.770920 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260809 08:46:15.899927 24348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:15.899693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:37217)
0809 08:46:15.899817 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:15.899863 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":55,"threads_started":1}
I20260809 08:46:16.707206 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:16.706976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:56287)
0809 08:46:16.707116 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:16.707158 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260809 08:46:16.771710 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:16.771480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:49377)
0809 08:46:16.771611 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:16.771665 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260809 08:46:16.900734 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:16.900452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:60515)
0809 08:46:16.900640 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:16.900687 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":76,"threads_started":1}
I20260809 08:46:17.179106 24356 raft_consensus.cc:482] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:17.179219 24356 raft_consensus.cc:504] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:46:17.179441 24356 leader_election.cc:288] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367), 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339)
W20260809 08:46:17.179986 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
W20260809 08:46:17.180105 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
I20260809 08:46:17.180136 23745 leader_election.cc:302] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac207fc59f2644c086ecb956acf15f19; no voters: 5df8657b7106472a98abc8cdeffdc431, b636bf8d5f9a458f926f8936b6ebe054
I20260809 08:46:17.180224 24356 raft_consensus.cc:2726] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:46:17.707975 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:17.707729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:50235)
0809 08:46:17.707889 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:17.707929 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260809 08:46:17.772493 24359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:17.772231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:41161)
0809 08:46:17.772408 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:17.772448 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
I20260809 08:46:17.901672 24360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:17.901335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:42671)
0809 08:46:17.901497 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:17.901616 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
I20260809 08:46:18.708878 24362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:18.708594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:34107)
0809 08:46:18.708763 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:18.708830 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":69,"threads_started":1}
I20260809 08:46:18.773284 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:18.773043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:41767)
0809 08:46:18.773190 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:18.773235 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260809 08:46:18.902426 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:18.902179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:48611)
0809 08:46:18.902328 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:18.902375 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260809 08:46:19.437780 24366 raft_consensus.cc:482] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:19.437932 24366 raft_consensus.cc:504] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:46:19.438196 24366 leader_election.cc:288] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367), 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339)
W20260809 08:46:19.438622 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
W20260809 08:46:19.438736 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
I20260809 08:46:19.438771 23745 leader_election.cc:302] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac207fc59f2644c086ecb956acf15f19; no voters: 5df8657b7106472a98abc8cdeffdc431, b636bf8d5f9a458f926f8936b6ebe054
I20260809 08:46:19.438853 24366 raft_consensus.cc:2726] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:46:19.709705 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:19.709420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:33283)
0809 08:46:19.709612 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:19.709648 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":69,"threads_started":1}
I20260809 08:46:19.774084 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:19.773845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:43305)
0809 08:46:19.773995 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:19.774034 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260809 08:46:19.903258 24369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:19.902947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:42741)
0809 08:46:19.903121 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:19.903164 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260809 08:46:20.710446 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:20.710214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:54389)
0809 08:46:20.710361 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:20.710399 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260809 08:46:20.774891 24372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:20.774637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:48177)
0809 08:46:20.774809 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:20.774844 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260809 08:46:20.904044 24373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:20.903807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:58857)
0809 08:46:20.903950 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:20.903992 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260809 08:46:21.711338 24374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:21.711056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:43895)
0809 08:46:21.711240 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:21.711289 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260809 08:46:21.775735 24376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:21.775468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:41771)
0809 08:46:21.775615 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:21.775671 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260809 08:46:21.904816 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:21.904536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:47723)
0809 08:46:21.904730 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:21.904770 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260809 08:46:21.959805 24379 raft_consensus.cc:482] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:21.959964 24379 raft_consensus.cc:504] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:46:21.960186 24379 leader_election.cc:288] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367), 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339)
W20260809 08:46:21.960753 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
W20260809 08:46:21.960825 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
I20260809 08:46:21.960875 23745 leader_election.cc:302] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac207fc59f2644c086ecb956acf15f19; no voters: 5df8657b7106472a98abc8cdeffdc431, b636bf8d5f9a458f926f8936b6ebe054
I20260809 08:46:21.960997 24379 raft_consensus.cc:2726] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:46:22.712312 24380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:22.712003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:55357)
0809 08:46:22.712188 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:22.712255 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":79,"threads_started":1}
I20260809 08:46:22.776600 24381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:22.776303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:38475)
0809 08:46:22.776493 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:22.776547 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":69,"threads_started":1}
I20260809 08:46:22.905737 24383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:22.905436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:40793)
0809 08:46:22.905635 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:22.905688 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":109,"threads_started":1}
I20260809 08:46:23.713168 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:23.712927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:48709)
0809 08:46:23.713072 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:23.713115 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260809 08:46:23.777513 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:23.777187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:45731)
0809 08:46:23.777409 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:23.777464 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260809 08:46:23.906664 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:23.906406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:45805)
0809 08:46:23.906563 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:23.906605 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260809 08:46:24.714090 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:24.713813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:41823)
0809 08:46:24.713990 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:24.714039 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260809 08:46:24.778390 24390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:24.778097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:51133)
0809 08:46:24.778288 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:24.778339 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":121,"threads_started":1}
I20260809 08:46:24.907449 24391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:24.907155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:53497)
0809 08:46:24.907333 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:24.907376 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":110,"threads_started":1}
I20260809 08:46:25.714884 24392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:25.714617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:45611)
0809 08:46:25.714786 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:25.714833 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":72,"threads_started":1}
I20260809 08:46:25.779088 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:25.778870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:57475)
0809 08:46:25.778998 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:25.779039 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260809 08:46:25.908182 24396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:25.907944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:41085)
0809 08:46:25.908079 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:25.908135 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260809 08:46:26.174973 24398 raft_consensus.cc:482] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:26.175104 24398 raft_consensus.cc:504] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:46:26.175236 24398 leader_election.cc:288] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367), 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339)
W20260809 08:46:26.175734 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
W20260809 08:46:26.175824 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
I20260809 08:46:26.175882 23745 leader_election.cc:302] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac207fc59f2644c086ecb956acf15f19; no voters: 5df8657b7106472a98abc8cdeffdc431, b636bf8d5f9a458f926f8936b6ebe054
I20260809 08:46:26.175990 24398 raft_consensus.cc:2726] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:46:26.715847 24401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:26.715592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:56349)
0809 08:46:26.715749 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:26.715797 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260809 08:46:26.779791 24402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:26.779567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:43899)
0809 08:46:26.779700 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:26.779743 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260809 08:46:26.908949 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:26.908702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:36327)
0809 08:46:26.908857 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:26.908901 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260809 08:46:27.716650 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:27.716405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:33347)
0809 08:46:27.716549 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:27.716602 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260809 08:46:27.780614 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:27.780346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:60985)
0809 08:46:27.780520 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:27.780563 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":105,"threads_started":1}
I20260809 08:46:27.909919 24408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:27.909656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:56313)
0809 08:46:27.909810 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:27.909862 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260809 08:46:28.143910 24411 raft_consensus.cc:482] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:28.144033 24411 raft_consensus.cc:504] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b636bf8d5f9a458f926f8936b6ebe054" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41367 } } peers { permanent_uuid: "5df8657b7106472a98abc8cdeffdc431" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40339 } } peers { permanent_uuid: "ac207fc59f2644c086ecb956acf15f19" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41003 } }
I20260809 08:46:28.144258 24411 leader_election.cc:288] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367), 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339)
W20260809 08:46:28.144644 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b636bf8d5f9a458f926f8936b6ebe054 (127.31.63.66:41367): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
W20260809 08:46:28.144729 23745 leader_election.cc:341] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5df8657b7106472a98abc8cdeffdc431 (127.31.63.67:40339): Not found: Tablet not found: f5ff45f8a3af4a3a9adac38ff1aea5bf
I20260809 08:46:28.144774 23745 leader_election.cc:302] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac207fc59f2644c086ecb956acf15f19; no voters: 5df8657b7106472a98abc8cdeffdc431, b636bf8d5f9a458f926f8936b6ebe054
I20260809 08:46:28.144853 24411 raft_consensus.cc:2726] T f5ff45f8a3af4a3a9adac38ff1aea5bf P ac207fc59f2644c086ecb956acf15f19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:46:28.717484 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:28.717241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.65:60375)
0809 08:46:28.717398 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:28.717438 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":65,"threads_started":1}
I20260809 08:46:28.781383 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:28.781147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.66:52359)
0809 08:46:28.781295 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:28.781334 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260809 08:46:28.906706 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 23677
I20260809 08:46:28.915426 24415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:28.915151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:37669 (local address 127.31.63.67:58405)
0809 08:46:28.915304 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:28.915358 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.126:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":60,"threads_started":1}
I20260809 08:46:28.920822 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 23933
I20260809 08:46:28.930546 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 24127
2026-08-09T08:46:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58880 ms)
[----------] 30 tests from SecurityITest (270159 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:28 dist-test-slave-gkw7 krb5kdc[24419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:28 dist-test-slave-gkw7 krb5kdc[24419](info): set up 2 sockets
Aug 09 08:46:28 dist-test-slave-gkw7 krb5kdc[24419](info): commencing operation
krb5kdc: starting...
W20260809 08:46:30.171013 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:30 dist-test-slave-gkw7 krb5kdc[24419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:46:30.298285 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:38683
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:38683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:30.381247 24436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:30.381518 24436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:30.381552 24436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:30.382788 24436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:30.382829 24436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:30.382844 24436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:30.382856 24436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:30.382896 24436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:30.384392 24436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:38683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:38683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:30.384651 24436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:30.384894 24436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:30.389846 24449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:30.389868 24446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:30.389853 24447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:30.390110 24436 server_base.cc:1034] running on GCE node
I20260809 08:46:30.390494 24436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:30.390753 24436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:30.391933 24436 hybrid_clock.cc:648] HybridClock initialized: now 1786265190391912 us; error 39 us; skew 500 ppm
Aug 09 08:46:30 dist-test-slave-gkw7 krb5kdc[24419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265190, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:30.394913 24436 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:46:30.395524 24436 webserver.cc:460] Webserver started at http://127.31.63.126:34853/ using document root <none> and password file <none>
I20260809 08:46:30.395905 24436 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:30.395998 24436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:30.396164 24436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:30.397282 24436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "6f54a3832502480386f70a2f6c8f9af0"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "fcfde81e5cd40d086d7413605d35e16d"
server_key_iv: "d99b75c690ff2afc1eb6740b8f1088b4"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.397709 24436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "6f54a3832502480386f70a2f6c8f9af0"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "fcfde81e5cd40d086d7413605d35e16d"
server_key_iv: "d99b75c690ff2afc1eb6740b8f1088b4"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.399119 24436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:30.399855 24463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:30.400059 24436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:30.400146 24436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "6f54a3832502480386f70a2f6c8f9af0"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "fcfde81e5cd40d086d7413605d35e16d"
server_key_iv: "d99b75c690ff2afc1eb6740b8f1088b4"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.400197 24436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:30.422837 24436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:30.425351 24436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:30.425472 24436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:30.429430 24436 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:38683
I20260809 08:46:30.429517 24538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:38683 every 8 connection(s)
I20260809 08:46:30.429883 24436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:30.430907 24539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:30.433363 24539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0: Bootstrap starting.
I20260809 08:46:30.433966 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 24436
I20260809 08:46:30.434044 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:30.434074 24539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:30.434278 31997 external_mini_cluster.cc:1428] Setting key d6d7c23476fe2722475e394a771fcb47
I20260809 08:46:30.434461 24539 log.cc:824] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:30.435359 24539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0: No bootstrap required, opened a new log
I20260809 08:46:30.437975 24539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54a3832502480386f70a2f6c8f9af0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38683 } }
I20260809 08:46:30.438153 24539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:30.438194 24539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f54a3832502480386f70a2f6c8f9af0, State: Initialized, Role: FOLLOWER
I20260809 08:46:30.438374 24539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54a3832502480386f70a2f6c8f9af0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38683 } }
I20260809 08:46:30.438481 24539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:30.438525 24539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:30.438592 24539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:30.439241 24539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54a3832502480386f70a2f6c8f9af0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38683 } }
I20260809 08:46:30.439357 24539 leader_election.cc:302] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f54a3832502480386f70a2f6c8f9af0; no voters:
I20260809 08:46:30.439541 24539 leader_election.cc:288] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:30.439574 24549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:30.439790 24549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [term 1 LEADER]: Becoming Leader. State: Replica: 6f54a3832502480386f70a2f6c8f9af0, State: Running, Role: LEADER
I20260809 08:46:30.439826 24539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:30.441336 24549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54a3832502480386f70a2f6c8f9af0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38683 } }
I20260809 08:46:30.442005 24551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f54a3832502480386f70a2f6c8f9af0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54a3832502480386f70a2f6c8f9af0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38683 } } }
I20260809 08:46:30.442013 24550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f54a3832502480386f70a2f6c8f9af0. Latest consensus state: current_term: 1 leader_uuid: "6f54a3832502480386f70a2f6c8f9af0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f54a3832502480386f70a2f6c8f9af0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38683 } } }
I20260809 08:46:30.442121 24551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:30.442145 24550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0 [sys.catalog]: This master's current role is: LEADER
Aug 09 08:46:30 dist-test-slave-gkw7 krb5kdc[24419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:30.442454 24564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:30.443027 24564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:30.444710 24564 catalog_manager.cc:1347] Generated new cluster ID: 4480bcbce45d4350852a8e25f70e94da
I20260809 08:46:30.444757 24564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:30.446885 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.434845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59502 (local address 127.31.63.126:38683)
0809 08:46:30.435013 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:30.435022 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:30.435070 (+ 48us) server_negotiation.cc:407] Connection header received
0809 08:46:30.435744 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:30.435754 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:30.435820 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:30.435984 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:30.436630 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.438177 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.439112 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.439745 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.440214 (+ 469us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.443050 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:30.443054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:30.443054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:30.443065 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:30.445145 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:30.445632 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:30.445635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:30.445636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:30.445736 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:30.445970 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:30.445971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:30.445971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:30.446245 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:30.446384 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:30.446542 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 08:46:30.446680 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260809 08:46:30.449847 24564 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:30.450344 24564 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:30.453434 24564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0: Generated new TSK 0
I20260809 08:46:30.453663 24564 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:46:30.484494 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38683
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:30.570298 24575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:30.570497 24575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:30.570519 24575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:30.571672 24575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:30.571713 24575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:30.571738 24575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:46:30.573205 24575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:30.573437 24575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:30.573719 24575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:30.579022 24582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:30.579062 24586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:30.579144 24581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:30.579046 24575 server_base.cc:1034] running on GCE node
I20260809 08:46:30.579458 24575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:30.579736 24575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:30.580897 24575 hybrid_clock.cc:648] HybridClock initialized: now 1786265190580889 us; error 33 us; skew 500 ppm
Aug 09 08:46:30 dist-test-slave-gkw7 krb5kdc[24419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265190, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:30.583650 24575 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:46:30.584074 24575 webserver.cc:460] Webserver started at http://127.31.63.65:43549/ using document root <none> and password file <none>
I20260809 08:46:30.584304 24575 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:30.584374 24575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:30.584529 24575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:30.585678 24575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "29cad938dc9442b3ab49cdb22e0daad3"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "786b54ace1a046662bd0bbb260486189"
server_key_iv: "6ee97caba6938a6fb99a8c00d23e1577"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.586058 24575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "29cad938dc9442b3ab49cdb22e0daad3"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "786b54ace1a046662bd0bbb260486189"
server_key_iv: "6ee97caba6938a6fb99a8c00d23e1577"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.587423 24575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:46:30.588176 24594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:30.588363 24575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:46:30.588470 24575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "29cad938dc9442b3ab49cdb22e0daad3"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "786b54ace1a046662bd0bbb260486189"
server_key_iv: "6ee97caba6938a6fb99a8c00d23e1577"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.588546 24575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:30.607738 24575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:30.609740 24575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:30.609902 24575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:30.610176 24575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:30.610515 24575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:30.610571 24575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:30.610617 24575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:30.610663 24575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:30.616865 24575 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:39397
I20260809 08:46:30.616952 24767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:39397 every 8 connection(s)
I20260809 08:46:30.617300 24575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:46:30.620882 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 24575
I20260809 08:46:30.621011 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:46:30.621215 31997 external_mini_cluster.cc:1428] Setting key 52417e86cb8a6c4c01fa91984a624ba3
Aug 09 08:46:30 dist-test-slave-gkw7 krb5kdc[24419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265190, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:30.628016 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.618118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:56623 (local address 127.31.63.126:38683)
0809 08:46:30.618169 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:30.618171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:30.618350 (+ 179us) server_negotiation.cc:407] Connection header received
0809 08:46:30.618858 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:30.618859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:30.618870 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:30.618933 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:30.619935 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.620588 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.622018 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.622349 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.622431 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.624833 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:30.624835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:30.624835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:30.624843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:30.626555 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:30.626973 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:30.626975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:30.626975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:30.627048 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:30.627310 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:30.627311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:30.627312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:30.627447 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:30.627539 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:30.627831 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 08:46:30.627925 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:46:30.628217 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.618026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:38683 (local address 127.31.63.65:56623)
0809 08:46:30.618205 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:30.618248 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:30.618766 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:30.619014 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:30.619018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:30.619639 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:30.619836 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.619841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.620677 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.620678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.621872 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.621875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.622445 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.622445 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.622558 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.623137 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:30.623146 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:30.624729 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:30.626656 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:30.626663 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:30.626665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:30.626886 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:30.627122 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:30.627122 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:30.627122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:30.627222 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:30.627536 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:30.627545 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:30.627688 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 08:46:30.627795 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 08:46:30.627958 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260809 08:46:30.628875 24769 heartbeater.cc:347] Connected to a master server at 127.31.63.126:38683
I20260809 08:46:30.628997 24769 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:30.629225 24769 heartbeater.cc:511] Master 127.31.63.126:38683 requested a full tablet report, sending...
I20260809 08:46:30.629809 24488 ts_manager.cc:194] Registered new tserver with Master: 29cad938dc9442b3ab49cdb22e0daad3 (127.31.63.65:39397)
I20260809 08:46:30.630427 24488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:56623
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:46:30.652187 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38683
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:30.737913 24778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:30.738127 24778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:30.738152 24778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:30.739307 24778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:30.739349 24778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:30.739404 24778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:46:30.741029 24778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:30.741278 24778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:30.741518 24778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:30.747295 24785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:30.747294 24787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:30.747535 24778 server_base.cc:1034] running on GCE node
W20260809 08:46:30.747303 24790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:30.747814 24778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:30.748015 24778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:30.749158 24778 hybrid_clock.cc:648] HybridClock initialized: now 1786265190749123 us; error 49 us; skew 500 ppm
Aug 09 08:46:30 dist-test-slave-gkw7 krb5kdc[24419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265190, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:30.751839 24778 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:46:30.752314 24778 webserver.cc:460] Webserver started at http://127.31.63.66:42959/ using document root <none> and password file <none>
I20260809 08:46:30.752501 24778 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:30.752542 24778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:30.752636 24778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:30.753719 24778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "d7b130ff55204797887c5a36dbed201b"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "ec9a63f7bf96caa0f58db11b55cee0f9"
server_key_iv: "4b2376da8b1a9327c22c3238e3eb98ba"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.754032 24778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "d7b130ff55204797887c5a36dbed201b"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "ec9a63f7bf96caa0f58db11b55cee0f9"
server_key_iv: "4b2376da8b1a9327c22c3238e3eb98ba"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.755330 24778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:30.756110 24799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:30.756371 24778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:30.756448 24778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "d7b130ff55204797887c5a36dbed201b"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "ec9a63f7bf96caa0f58db11b55cee0f9"
server_key_iv: "4b2376da8b1a9327c22c3238e3eb98ba"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.756533 24778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:30.779435 24778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:30.781761 24778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:30.781930 24778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:30.782222 24778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:30.782567 24778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:30.782609 24778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:30.782632 24778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:30.782676 24778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:30.789283 24778 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:33609
I20260809 08:46:30.789453 24962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:33609 every 8 connection(s)
I20260809 08:46:30.789727 24778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:46:30 dist-test-slave-gkw7 krb5kdc[24419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265190, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:30.798326 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 24778
I20260809 08:46:30.798415 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:46:30.798657 31997 external_mini_cluster.cc:1428] Setting key c6b049dd95bce08adfa79b317fe4cad3
I20260809 08:46:30.799964 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.790532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:32843 (local address 127.31.63.126:38683)
0809 08:46:30.790595 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:30.790598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:30.790797 (+ 199us) server_negotiation.cc:407] Connection header received
0809 08:46:30.791429 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:30.791431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:30.791443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:30.791505 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:30.792546 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.793084 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.794310 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.794557 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.794634 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.796895 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:30.796896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:30.796896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:30.796903 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:30.798362 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:30.798844 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:30.798846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:30.798847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:30.798907 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:30.799224 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:30.799226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:30.799226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:30.799410 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:30.799484 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:30.799786 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 08:46:30.799872 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:46:30.800174 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.790515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:38683 (local address 127.31.63.66:32843)
0809 08:46:30.790682 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:30.790727 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:30.791300 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:30.791584 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:30.791588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:30.792193 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:30.792434 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.792441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.793199 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.793200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.794207 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.794210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.794641 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.794642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.794747 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.795233 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:30.795242 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:30.796779 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:30.798470 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:30.798477 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:30.798479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:30.798780 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:30.799000 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:30.799001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:30.799001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:30.799130 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:30.799484 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:30.799493 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:30.799636 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 08:46:30.799746 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 08:46:30.799921 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260809 08:46:30.800707 24963 heartbeater.cc:347] Connected to a master server at 127.31.63.126:38683
I20260809 08:46:30.800820 24963 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:30.801024 24963 heartbeater.cc:511] Master 127.31.63.126:38683 requested a full tablet report, sending...
I20260809 08:46:30.801438 24488 ts_manager.cc:194] Registered new tserver with Master: d7b130ff55204797887c5a36dbed201b (127.31.63.66:33609)
I20260809 08:46:30.802032 24488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:32843
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:46:30.827018 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38683
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:30.911424 24970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:30.911664 24970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:30.911702 24970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:30.912838 24970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:30.912905 24970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:30.912952 24970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:46:30.914551 24970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:30.914825 24970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:30.915071 24970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:30.920274 24980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:30.920311 24977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:30.920365 24970 server_base.cc:1034] running on GCE node
W20260809 08:46:30.920306 24978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:30.920745 24970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:30.921043 24970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:30.922209 24970 hybrid_clock.cc:648] HybridClock initialized: now 1786265190922204 us; error 38 us; skew 500 ppm
Aug 09 08:46:30 dist-test-slave-gkw7 krb5kdc[24419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265190, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:30.931530 24970 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:46:30.931979 24970 webserver.cc:460] Webserver started at http://127.31.63.67:40697/ using document root <none> and password file <none>
I20260809 08:46:30.932313 24970 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:30.932386 24970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:30.932556 24970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:30.933709 24970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "384643cf716b49a8ae848175d3309e90"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "7e02882139435a3bfcaaebbdc6b8ce12"
server_key_iv: "56cdba46f52d1ff1e3db1cc1c5a26acb"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.934082 24970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "384643cf716b49a8ae848175d3309e90"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "7e02882139435a3bfcaaebbdc6b8ce12"
server_key_iv: "56cdba46f52d1ff1e3db1cc1c5a26acb"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.935469 24970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:46:30.936241 24989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:30.936476 24970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:30.936597 24970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "384643cf716b49a8ae848175d3309e90"
format_stamp: "Formatted at 2026-08-09 08:46:30 on dist-test-slave-gkw7"
server_key: "7e02882139435a3bfcaaebbdc6b8ce12"
server_key_iv: "56cdba46f52d1ff1e3db1cc1c5a26acb"
server_key_version: "encryptionkey@0"
I20260809 08:46:30.936692 24970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:30.949860 24970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:30.951885 24970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:30.952003 24970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:30.952186 24970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:30.952611 24970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:30.952648 24970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:30.952672 24970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:30.952684 24970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:30.959312 24970 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:40491
I20260809 08:46:30.959348 25149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:40491 every 8 connection(s)
I20260809 08:46:30.959697 24970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:46:30.962690 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 24970
I20260809 08:46:30.962788 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:46:30.962963 31997 external_mini_cluster.cc:1428] Setting key 5428a20b13697011d680c197ec92e438
Aug 09 08:46:30 dist-test-slave-gkw7 krb5kdc[24419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265190, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:30.969769 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.960608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:43637 (local address 127.31.63.126:38683)
0809 08:46:30.960695 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:30.960697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:30.960825 (+ 128us) server_negotiation.cc:407] Connection header received
0809 08:46:30.961358 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:30.961360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:30.961369 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:30.961411 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:30.962472 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.963266 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.964545 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.964742 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.964802 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.967127 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:30.967129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:30.967129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:30.967135 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:30.968409 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:30.968821 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:30.968822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:30.968822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:30.968876 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:30.969123 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:30.969124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:30.969124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:30.969227 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:30.969281 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:30.969640 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 08:46:30.969705 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 08:46:30.970089 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.960516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:38683 (local address 127.31.63.67:43637)
0809 08:46:30.960682 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:30.960738 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:30.961267 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:30.961463 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:30.961467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:30.962173 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:30.962376 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.962381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.963373 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.963375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.964460 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.964464 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.964826 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.964827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.964941 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.965583 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:30.965593 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:30.967029 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:30.968498 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:30.968505 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:30.968507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:30.968747 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:30.968941 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:30.968942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:30.968942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:30.969055 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:30.969311 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:30.969323 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:30.969445 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 08:46:30.969636 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 08:46:30.969782 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260809 08:46:30.970606 25151 heartbeater.cc:347] Connected to a master server at 127.31.63.126:38683
I20260809 08:46:30.970722 25151 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:30.970942 25151 heartbeater.cc:511] Master 127.31.63.126:38683 requested a full tablet report, sending...
I20260809 08:46:30.971374 24488 ts_manager.cc:194] Registered new tserver with Master: 384643cf716b49a8ae848175d3309e90 (127.31.63.67:40491)
I20260809 08:46:30.971758 24488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:43637
I20260809 08:46:30.974903 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:46:30.982268 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.976229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59518 (local address 127.31.63.126:38683)
0809 08:46:30.976274 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:30.976276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:30.976423 (+ 147us) server_negotiation.cc:407] Connection header received
0809 08:46:30.976505 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:30.976505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:30.976515 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:30.976585 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:30.977155 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.977682 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.978419 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.978610 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.978683 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.979856 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:30.979857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:30.979858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:30.979865 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:30.980991 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:30.981394 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:30.981395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:30.981396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:30.981456 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:30.981692 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:30.981693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:30.981694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:30.981856 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:30.981966 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:30.982082 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 08:46:30.982166 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:46:30.983887 24488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:46:30.985518 24488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:46:30.992133 25173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.988176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:33609 (local address 127.0.0.1:39838)
0809 08:46:30.988457 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:30.988467 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:30.988522 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:30.988741 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:30.988744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:30.988746 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:30.988816 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.988818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.989816 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.989818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.990903 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.990906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.991820 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.991821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.991904 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.991919 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 08:46:30.991989 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:46:30.992045 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":66,"threads_started":1}
I20260809 08:46:30.992154 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.988313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.31.63.66:33609)
0809 08:46:30.988551 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:30.988557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:30.988567 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:46:30.988619 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:30.988621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:30.988637 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:30.988716 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:30.988887 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.989715 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.990983 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.991732 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.991805 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.991861 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:30.992019 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 08:46:30.992067 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":116,"threads_started":1}
I20260809 08:46:30.992733 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.988332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40491 (local address 127.0.0.1:37300)
0809 08:46:30.988468 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:30.988479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:30.988532 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:30.989148 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:30.989149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:30.989151 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:30.989202 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.989203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.990434 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.990435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.991378 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.991380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.992513 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.992514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.992568 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.992572 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:30.992608 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 08:46:30.992663 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260809 08:46:30.993072 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.988455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.31.63.67:40491)
0809 08:46:30.988989 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:30.988993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:30.989002 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:46:30.989045 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:30.989046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:30.989059 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:30.989150 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:30.989261 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.990312 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.991456 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.992145 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.992862 (+ 717us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.992928 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:30.992954 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 08:46:30.992993 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":93,"threads_started":1}
I20260809 08:46:30.993165 25170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.987944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:39397 (local address 127.0.0.1:33932)
0809 08:46:30.988111 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:30.988145 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:30.988239 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:30.989604 (+ 1365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:30.989605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:30.989607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:30.989646 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.989647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.990896 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.990897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.991977 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:30.991981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.992929 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.992931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:30.992994 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.992998 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:30.993040 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 08:46:30.993093 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260809 08:46:30.993158 24862 tablet_service.cc:1467] Processing CreateTablet for tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064 (DEFAULT_TABLE table=test-table [id=15a80b2b87d44ca4bb45de6c1f0c214c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:30.993602 24862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:30.993613 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:30.988077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33932 (local address 127.31.63.65:39397)
0809 08:46:30.988683 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:30.988688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:30.988699 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:46:30.988741 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:30.988743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:30.988760 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:30.988871 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:30.989730 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.990781 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.992071 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:30.992824 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:30.993354 (+ 530us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:30.993422 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:30.993454 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 08:46:30.993486 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":108,"threads_started":1}
I20260809 08:46:30.993969 25057 tablet_service.cc:1467] Processing CreateTablet for tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064 (DEFAULT_TABLE table=test-table [id=15a80b2b87d44ca4bb45de6c1f0c214c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:30.994186 24668 tablet_service.cc:1467] Processing CreateTablet for tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064 (DEFAULT_TABLE table=test-table [id=15a80b2b87d44ca4bb45de6c1f0c214c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:30.994300 25057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:30.994488 24668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:30.996345 25178 tablet_bootstrap.cc:492] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: Bootstrap starting.
I20260809 08:46:30.997026 25178 tablet_bootstrap.cc:654] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:30.997637 25178 log.cc:824] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:30.997679 25179 tablet_bootstrap.cc:492] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: Bootstrap starting.
I20260809 08:46:30.997886 25182 tablet_bootstrap.cc:492] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3: Bootstrap starting.
I20260809 08:46:30.998435 25179 tablet_bootstrap.cc:654] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:30.998548 25178 tablet_bootstrap.cc:492] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: No bootstrap required, opened a new log
I20260809 08:46:30.998592 25182 tablet_bootstrap.cc:654] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:30.998637 25178 ts_tablet_manager.cc:1397] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:46:30.999078 25179 log.cc:824] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:30.999132 25182 log.cc:824] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:30.999821 25182 tablet_bootstrap.cc:492] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3: No bootstrap required, opened a new log
I20260809 08:46:30.999878 25179 tablet_bootstrap.cc:492] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: No bootstrap required, opened a new log
I20260809 08:46:30.999912 25182 ts_tablet_manager.cc:1397] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:46:30.999997 25179 ts_tablet_manager.cc:1397] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:46:31.001052 25178 raft_consensus.cc:348] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cad938dc9442b3ab49cdb22e0daad3" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 39397 } } peers { permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 } } peers { permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 } }
I20260809 08:46:31.001245 25178 raft_consensus.cc:374] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:31.001282 25178 raft_consensus.cc:729] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b130ff55204797887c5a36dbed201b, State: Initialized, Role: FOLLOWER
I20260809 08:46:31.001434 25178 consensus_queue.cc:260] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cad938dc9442b3ab49cdb22e0daad3" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 39397 } } peers { permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 } } peers { permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 } }
I20260809 08:46:31.001379 25182 raft_consensus.cc:348] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cad938dc9442b3ab49cdb22e0daad3" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 39397 } } peers { permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 } } peers { permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 } }
I20260809 08:46:31.001609 25182 raft_consensus.cc:374] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:31.001650 25182 raft_consensus.cc:729] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29cad938dc9442b3ab49cdb22e0daad3, State: Initialized, Role: FOLLOWER
I20260809 08:46:31.001755 25178 ts_tablet_manager.cc:1428] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 08:46:31.001776 25182 consensus_queue.cc:260] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cad938dc9442b3ab49cdb22e0daad3" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 39397 } } peers { permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 } } peers { permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 } }
I20260809 08:46:31.001852 24963 heartbeater.cc:503] Master 127.31.63.126:38683 was elected leader, sending a full tablet report...
I20260809 08:46:31.002090 25182 ts_tablet_manager.cc:1428] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:46:31.002246 24769 heartbeater.cc:503] Master 127.31.63.126:38683 was elected leader, sending a full tablet report...
I20260809 08:46:31.002126 25179 raft_consensus.cc:348] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cad938dc9442b3ab49cdb22e0daad3" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 39397 } } peers { permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 } } peers { permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 } }
I20260809 08:46:31.002290 25179 raft_consensus.cc:374] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:31.002372 25179 raft_consensus.cc:729] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 384643cf716b49a8ae848175d3309e90, State: Initialized, Role: FOLLOWER
I20260809 08:46:31.002511 25179 consensus_queue.cc:260] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cad938dc9442b3ab49cdb22e0daad3" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 39397 } } peers { permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 } } peers { permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 } }
I20260809 08:46:31.002801 25179 ts_tablet_manager.cc:1428] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:46:31.002866 25151 heartbeater.cc:503] Master 127.31.63.126:38683 was elected leader, sending a full tablet report...
W20260809 08:46:31.040407 24965 tablet.cc:2369] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:46:31.115557 25189 raft_consensus.cc:482] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:31.115747 25189 raft_consensus.cc:504] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cad938dc9442b3ab49cdb22e0daad3" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 39397 } } peers { permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 } } peers { permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 } }
I20260809 08:46:31.116179 25189 leader_election.cc:288] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7b130ff55204797887c5a36dbed201b (127.31.63.66:33609), 384643cf716b49a8ae848175d3309e90 (127.31.63.67:40491)
W20260809 08:46:31.118100 24770 tablet.cc:2369] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:46:31.119462 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.116591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:52019 (local address 127.31.63.66:33609)
0809 08:46:31.116646 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.116648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.116659 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:46:31.116695 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.116696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.116707 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.116781 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:31.116978 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.117477 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.118683 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.119084 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.119170 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.119180 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.119345 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.119386 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:46:31.119499 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.116761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:56743 (local address 127.31.63.67:40491)
0809 08:46:31.116824 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.116826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.116835 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:46:31.116891 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.116892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.116902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.116984 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:31.117166 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.117711 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.118735 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.119137 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.119227 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.119236 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.119383 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.119423 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:46:31.119496 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.116448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:33609 (local address 127.31.63.65:52019)
0809 08:46:31.116516 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.116530 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:31.116594 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:31.116807 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:31.116809 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:31.116812 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:31.116887 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.116889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.117627 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.117628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.118577 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.118580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.119168 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.119169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.119267 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.119270 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:46:31.119339 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:46:31.119382 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 08:46:31.119649 25192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.116580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40491 (local address 127.31.63.65:56743)
0809 08:46:31.116777 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.116787 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:31.116835 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:31.116989 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:31.116990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:31.116991 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:31.117076 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.117078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.117799 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.117800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.118642 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.118645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.119228 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.119229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.119305 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.119308 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:46:31.119377 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:46:31.119419 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260809 08:46:31.119864 24894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bb5fb2cb1d148aa8c3171c6cb4ef064" candidate_uuid: "29cad938dc9442b3ab49cdb22e0daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7b130ff55204797887c5a36dbed201b" is_pre_election: true
I20260809 08:46:31.119979 25086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bb5fb2cb1d148aa8c3171c6cb4ef064" candidate_uuid: "29cad938dc9442b3ab49cdb22e0daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "384643cf716b49a8ae848175d3309e90" is_pre_election: true
I20260809 08:46:31.120066 24894 raft_consensus.cc:2454] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29cad938dc9442b3ab49cdb22e0daad3 in term 0.
I20260809 08:46:31.120177 25086 raft_consensus.cc:2454] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29cad938dc9442b3ab49cdb22e0daad3 in term 0.
I20260809 08:46:31.120352 24595 leader_election.cc:302] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cad938dc9442b3ab49cdb22e0daad3, d7b130ff55204797887c5a36dbed201b; no voters:
I20260809 08:46:31.120478 25189 raft_consensus.cc:2781] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:46:31.120540 25189 raft_consensus.cc:482] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:46:31.120563 25189 raft_consensus.cc:3037] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:31.121326 25189 raft_consensus.cc:504] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cad938dc9442b3ab49cdb22e0daad3" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 39397 } } peers { permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 } } peers { permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 } }
I20260809 08:46:31.121500 25189 leader_election.cc:288] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [CANDIDATE]: Term 1 election: Requested vote from peers d7b130ff55204797887c5a36dbed201b (127.31.63.66:33609), 384643cf716b49a8ae848175d3309e90 (127.31.63.67:40491)
I20260809 08:46:31.121724 24894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bb5fb2cb1d148aa8c3171c6cb4ef064" candidate_uuid: "29cad938dc9442b3ab49cdb22e0daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7b130ff55204797887c5a36dbed201b"
I20260809 08:46:31.121819 25086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bb5fb2cb1d148aa8c3171c6cb4ef064" candidate_uuid: "29cad938dc9442b3ab49cdb22e0daad3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "384643cf716b49a8ae848175d3309e90"
I20260809 08:46:31.121837 24894 raft_consensus.cc:3037] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:31.121901 25086 raft_consensus.cc:3037] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:31.122818 24894 raft_consensus.cc:2454] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29cad938dc9442b3ab49cdb22e0daad3 in term 1.
I20260809 08:46:31.122818 25086 raft_consensus.cc:2454] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29cad938dc9442b3ab49cdb22e0daad3 in term 1.
I20260809 08:46:31.123000 24595 leader_election.cc:302] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cad938dc9442b3ab49cdb22e0daad3, 384643cf716b49a8ae848175d3309e90; no voters:
I20260809 08:46:31.123103 25189 raft_consensus.cc:2781] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:31.123271 25189 raft_consensus.cc:686] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [term 1 LEADER]: Becoming Leader. State: Replica: 29cad938dc9442b3ab49cdb22e0daad3, State: Running, Role: LEADER
I20260809 08:46:31.123375 25189 consensus_queue.cc:237] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cad938dc9442b3ab49cdb22e0daad3" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 39397 } } peers { permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 } } peers { permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 } }
I20260809 08:46:31.124224 24486 catalog_manager.cc:5495] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29cad938dc9442b3ab49cdb22e0daad3 (127.31.63.65). New cstate: current_term: 1 leader_uuid: "29cad938dc9442b3ab49cdb22e0daad3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29cad938dc9442b3ab49cdb22e0daad3" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 39397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 } health_report { overall_health: UNKNOWN } } }
I20260809 08:46:31.136974 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.134758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33948 (local address 127.31.63.65:39397)
0809 08:46:31.134818 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.134820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.134828 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:46:31.134879 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.134880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.134898 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.134949 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:46:31.135065 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.135555 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.136286 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.136476 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.136532 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.136673 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:46:31.136773 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:46:31.136834 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.136866 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.136884 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:46:31.139041 25086 raft_consensus.cc:1261] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90 [term 1 FOLLOWER]: Refusing update from remote peer 29cad938dc9442b3ab49cdb22e0daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:31.139041 24894 raft_consensus.cc:1261] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b [term 1 FOLLOWER]: Refusing update from remote peer 29cad938dc9442b3ab49cdb22e0daad3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:31.139487 25194 consensus_queue.cc:1035] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7b130ff55204797887c5a36dbed201b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 33609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:31.139681 25189 consensus_queue.cc:1035] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 29cad938dc9442b3ab49cdb22e0daad3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "384643cf716b49a8ae848175d3309e90" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:31.142804 25202 mvcc.cc:205] Tried to move back new op lower bound from 7316542222902194176 to 7316542222841901056. Current Snapshot: MvccSnapshot[applied={T|T < 7316542222902194176}]
I20260809 08:46:31.143610 25204 mvcc.cc:205] Tried to move back new op lower bound from 7316542222902194176 to 7316542222841901056. Current Snapshot: MvccSnapshot[applied={T|T < 7316542222902194176}]
I20260809 08:46:31.144925 24486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:31.145036 24486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:31.145900 24486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f54a3832502480386f70a2f6c8f9af0: Sending DeleteTablet for 3 replicas of tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064
I20260809 08:46:31.146288 24668 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:31 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:33932
I20260809 08:46:31.146358 25057 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:31 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:37300
I20260809 08:46:31.146534 24862 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:31 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:39838
I20260809 08:46:31.146625 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 24575
I20260809 08:46:31.146850 25220 tablet_replica.cc:330] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: stopping tablet replica
I20260809 08:46:31.151872 25220 raft_consensus.cc:2229] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:31.152698 25220 raft_consensus.cc:2258] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:31.152714 25219 tablet_replica.cc:330] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: stopping tablet replica
I20260809 08:46:31.152964 25219 raft_consensus.cc:2229] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:31.153191 25219 raft_consensus.cc:2258] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:31.155472 25220 ts_tablet_manager.cc:1905] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:31.157137 25219 ts_tablet_manager.cc:1905] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:31.157465 25220 ts_tablet_manager.cc:1918] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:31.157656 25220 log.cc:1196] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/7bb5fb2cb1d148aa8c3171c6cb4ef064
I20260809 08:46:31.158026 25220 ts_tablet_manager.cc:1939] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P d7b130ff55204797887c5a36dbed201b: Deleting consensus metadata
I20260809 08:46:31.158490 24464 catalog_manager.cc:4841] TS d7b130ff55204797887c5a36dbed201b (127.31.63.66:33609): tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064 (table test-table [id=15a80b2b87d44ca4bb45de6c1f0c214c]) successfully deleted
I20260809 08:46:31.159389 25219 ts_tablet_manager.cc:1918] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:31.159482 25219 log.cc:1196] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/7bb5fb2cb1d148aa8c3171c6cb4ef064
I20260809 08:46:31.159811 25219 ts_tablet_manager.cc:1939] T 7bb5fb2cb1d148aa8c3171c6cb4ef064 P 384643cf716b49a8ae848175d3309e90: Deleting consensus metadata
I20260809 08:46:31.160280 24464 catalog_manager.cc:4841] TS 384643cf716b49a8ae848175d3309e90 (127.31.63.67:40491): tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064 (table test-table [id=15a80b2b87d44ca4bb45de6c1f0c214c]) successfully deleted
W20260809 08:46:31.162850 24464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:39397 (error 108)
I20260809 08:46:31.163194 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 24778
I20260809 08:46:31.163414 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.163290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:39397 (local address 127.0.0.1:33956)
0809 08:46:31.163343 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.163376 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:39397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 08:46:31.168246 24464 catalog_manager.cc:4584] TS 29cad938dc9442b3ab49cdb22e0daad3 (127.31.63.65:39397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7bb5fb2cb1d148aa8c3171c6cb4ef064: Network error: Client connection negotiation failed: client connection to 127.31.63.65:39397: connect: Connection refused (error 111)
I20260809 08:46:31.174165 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 24970
I20260809 08:46:31.204700 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 24436
2026-08-09T08:46:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2275 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T08:46:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:31Z Disabled control of system clock
I20260809 08:46:31.240248 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:46749
--webserver_interface=127.31.63.126
--webserver_port=0
--builtin_ntp_servers=127.31.63.84:35189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:46749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:46:31.326910 25228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:31.327136 25228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:31.327157 25228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:31.328241 25228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:31.328274 25228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:31.328286 25228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:31.328298 25228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:31.328310 25228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:31.329628 25228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:46749
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:46749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:31.329838 25228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:31.330073 25228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:31.335273 25233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:31.335312 25236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:31.335316 25234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:31.335481 25228 server_base.cc:1034] running on GCE node
I20260809 08:46:31.335789 25228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:31.336109 25228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:31.337283 25228 hybrid_clock.cc:648] HybridClock initialized: now 1786265191337259 us; error 41 us; skew 500 ppm
I20260809 08:46:31.338657 25228 webserver.cc:460] Webserver started at http://127.31.63.126:37859/ using document root <none> and password file <none>
I20260809 08:46:31.338927 25228 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:31.339005 25228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:31.339181 25228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:31.340277 25228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "dbe8def50775444a9cd92f1ca76b773a"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "9a52e427005022b9f4cbe0b2f128c4d4"
server_key_iv: "0ad2e304580b81099b3a7103dcb225d0"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.340659 25228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "dbe8def50775444a9cd92f1ca76b773a"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "9a52e427005022b9f4cbe0b2f128c4d4"
server_key_iv: "0ad2e304580b81099b3a7103dcb225d0"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.342303 25228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:31.343209 25242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:31.343423 25228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:31.343542 25228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "dbe8def50775444a9cd92f1ca76b773a"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "9a52e427005022b9f4cbe0b2f128c4d4"
server_key_iv: "0ad2e304580b81099b3a7103dcb225d0"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.343623 25228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:31.351972 25228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:31.352275 25228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:31.352427 25228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:31.356549 25228 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:46749
I20260809 08:46:31.356591 25321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:46749 every 8 connection(s)
I20260809 08:46:31.356964 25228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:31.358062 25324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:31.360359 25324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a: Bootstrap starting.
I20260809 08:46:31.360922 25324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:31.361183 25324 log.cc:824] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:31.361927 25324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a: No bootstrap required, opened a new log
I20260809 08:46:31.363497 25324 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe8def50775444a9cd92f1ca76b773a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46749 } }
I20260809 08:46:31.363643 25324 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:31.363667 25324 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe8def50775444a9cd92f1ca76b773a, State: Initialized, Role: FOLLOWER
I20260809 08:46:31.363807 25324 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe8def50775444a9cd92f1ca76b773a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46749 } }
I20260809 08:46:31.363888 25324 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:31.363914 25324 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:31.363941 25324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:31.364423 25324 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe8def50775444a9cd92f1ca76b773a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46749 } }
I20260809 08:46:31.364507 25324 leader_election.cc:302] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbe8def50775444a9cd92f1ca76b773a; no voters:
I20260809 08:46:31.364651 25324 leader_election.cc:288] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:31.364734 25332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:31.364898 25332 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [term 1 LEADER]: Becoming Leader. State: Replica: dbe8def50775444a9cd92f1ca76b773a, State: Running, Role: LEADER
I20260809 08:46:31.365058 25332 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe8def50775444a9cd92f1ca76b773a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46749 } }
I20260809 08:46:31.365216 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 25228
I20260809 08:46:31.365337 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:31.365454 25333 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbe8def50775444a9cd92f1ca76b773a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe8def50775444a9cd92f1ca76b773a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46749 } } }
I20260809 08:46:31.365607 31997 external_mini_cluster.cc:1428] Setting key b078ce0d2a7a0893dee1ca98db02eefe
I20260809 08:46:31.365592 25334 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbe8def50775444a9cd92f1ca76b773a. Latest consensus state: current_term: 1 leader_uuid: "dbe8def50775444a9cd92f1ca76b773a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbe8def50775444a9cd92f1ca76b773a" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46749 } } }
I20260809 08:46:31.365656 25333 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:31.365715 25334 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:31.364923 25324 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:31.366410 25336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:31.367013 25336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:31.368764 25336 catalog_manager.cc:1347] Generated new cluster ID: d4c4f66b47594e219ab5c59453eff30e
I20260809 08:46:31.368821 25336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:31.373111 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.366363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40802 (local address 127.31.63.126:46749)
0809 08:46:31.367534 (+ 1171us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.367542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.367576 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:46:31.367690 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.367698 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.367771 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.367932 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:31.368175 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.369617 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.370787 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.371419 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.371530 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.372296 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:31.372300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:31.372300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:31.372509 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:31.372569 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:31.372583 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:31.372643 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.372738 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.372905 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1094,"thread_start_us":103,"threads_started":1}
I20260809 08:46:31.377488 25336 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:31.378082 25336 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:31.380137 25336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a: Generated new TSK 0
I20260809 08:46:31.380326 25336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:31.382265 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46749
--builtin_ntp_servers=127.31.63.84:35189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:46:31.468817 25363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:31.469023 25363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:31.469046 25363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:31.470207 25363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:31.470247 25363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:31.470273 25363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:46:31.471645 25363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:31.471916 25363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:31.472254 25363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:31.477468 25374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:31.477455 25372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:31.477455 25371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:31.478029 25363 server_base.cc:1034] running on GCE node
I20260809 08:46:31.478255 25363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:31.478533 25363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:31.479727 25363 hybrid_clock.cc:648] HybridClock initialized: now 1786265191479704 us; error 41 us; skew 500 ppm
I20260809 08:46:31.481184 25363 webserver.cc:460] Webserver started at http://127.31.63.65:38727/ using document root <none> and password file <none>
I20260809 08:46:31.481467 25363 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:31.481550 25363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:31.481688 25363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:31.483086 25363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "d597a635a8654d53beb31067f9481ebb"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "57dea62bbcd38da9c4c4c5ebab3ba58d"
server_key_iv: "849ad71fc7b07c462edc984c518b0023"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.483490 25363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "d597a635a8654d53beb31067f9481ebb"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "57dea62bbcd38da9c4c4c5ebab3ba58d"
server_key_iv: "849ad71fc7b07c462edc984c518b0023"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.485222 25363 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 08:46:31.486922 25383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:31.487120 25363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:46:31.487215 25363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "d597a635a8654d53beb31067f9481ebb"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "57dea62bbcd38da9c4c4c5ebab3ba58d"
server_key_iv: "849ad71fc7b07c462edc984c518b0023"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.487300 25363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:31.498885 25363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:31.499243 25363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:31.499409 25363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:31.499665 25363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:31.500007 25363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:31.500041 25363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:31.500095 25363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:31.500120 25363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:31.506331 25363 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:37239
I20260809 08:46:31.506426 25540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:37239 every 8 connection(s)
I20260809 08:46:31.506769 25363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:46:31.508266 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 25363
I20260809 08:46:31.508388 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:46:31.508625 31997 external_mini_cluster.cc:1428] Setting key 7df48c0196f9a783eeeeefc181118fa7
I20260809 08:46:31.509763 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46749
--builtin_ntp_servers=127.31.63.84:35189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 08:46:31.512769 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.507554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:50329 (local address 127.31.63.126:46749)
0809 08:46:31.507617 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.507619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.507847 (+ 228us) server_negotiation.cc:407] Connection header received
0809 08:46:31.508436 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.508439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.508454 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.508529 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:31.509358 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.510123 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.511121 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.511457 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.511531 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.512490 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:31.512492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:31.512492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:31.512498 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:31.512513 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:31.512517 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:31.512561 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.512683 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.512709 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:46:31.513051 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.507512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:46749 (local address 127.31.63.65:50329)
0809 08:46:31.507717 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.507754 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:31.508286 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:31.508584 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:31.508589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:31.508597 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:31.509245 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.509251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.510223 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.510225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.511022 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.511024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.511558 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.511559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.511748 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.512324 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:46:31.512325 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:31.512366 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:46:31.512395 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:31.512585 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:31.512592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:31.512595 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:46:31.512688 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 08:46:31.512799 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260809 08:46:31.513727 25542 heartbeater.cc:347] Connected to a master server at 127.31.63.126:46749
I20260809 08:46:31.513823 25542 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:31.514026 25542 heartbeater.cc:511] Master 127.31.63.126:46749 requested a full tablet report, sending...
I20260809 08:46:31.514631 25266 ts_manager.cc:194] Registered new tserver with Master: d597a635a8654d53beb31067f9481ebb (127.31.63.65:37239)
I20260809 08:46:31.515245 25266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.65:50329
W20260809 08:46:31.595036 25546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:31.595285 25546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:31.595322 25546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:31.596453 25546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:31.596521 25546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:31.596565 25546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:46:31.597966 25546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:31.598265 25546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:31.598587 25546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:31.603912 25556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:31.603922 25551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:31.603931 25552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:31.604202 25546 server_base.cc:1034] running on GCE node
I20260809 08:46:31.604389 25546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:31.604588 25546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:31.605727 25546 hybrid_clock.cc:648] HybridClock initialized: now 1786265191605703 us; error 38 us; skew 500 ppm
I20260809 08:46:31.606930 25546 webserver.cc:460] Webserver started at http://127.31.63.66:35513/ using document root <none> and password file <none>
I20260809 08:46:31.607210 25546 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:31.607259 25546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:31.607434 25546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:31.608503 25546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "326bea9497ab4958b981f5456d1a6f2b"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "51daa8f1d03709e6f889c1b910cdc955"
server_key_iv: "6dae0292ac7c2355b05af9f5e871dbed"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.608917 25546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "326bea9497ab4958b981f5456d1a6f2b"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "51daa8f1d03709e6f889c1b910cdc955"
server_key_iv: "6dae0292ac7c2355b05af9f5e871dbed"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.610684 25546 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 08:46:31.611441 25565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:31.611675 25546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:31.611774 25546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "326bea9497ab4958b981f5456d1a6f2b"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "51daa8f1d03709e6f889c1b910cdc955"
server_key_iv: "6dae0292ac7c2355b05af9f5e871dbed"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.611841 25546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:31.639093 25546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:31.639432 25546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:31.639573 25546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:31.639820 25546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:31.640136 25546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:31.640174 25546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:31.640202 25546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:31.640219 25546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:31.647959 25546 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:45009
I20260809 08:46:31.648000 25734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:45009 every 8 connection(s)
I20260809 08:46:31.648422 25546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:46:31.653996 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.649319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:50099 (local address 127.31.63.126:46749)
0809 08:46:31.649388 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.649390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.649751 (+ 361us) server_negotiation.cc:407] Connection header received
0809 08:46:31.650254 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.650256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.650270 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.650341 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:31.651058 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.651610 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.652592 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.652788 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.652856 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.653753 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:31.653754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:31.653754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:31.653759 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:31.653774 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:31.653778 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:31.653826 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.653909 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.653934 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:46:31.654248 25738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.649331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:46749 (local address 127.31.63.66:50099)
0809 08:46:31.649595 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.649642 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:31.650130 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:31.650421 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:31.650425 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:31.650433 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:31.650946 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.650950 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.651701 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.651701 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.652499 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.652501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.652878 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.652879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.653003 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.653624 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:46:31.653624 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:31.653653 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:46:31.653675 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:31.653840 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:31.653845 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:31.653847 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:46:31.653899 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:46:31.653981 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":105,"threads_started":1}
I20260809 08:46:31.654804 25735 heartbeater.cc:347] Connected to a master server at 127.31.63.126:46749
I20260809 08:46:31.654911 25735 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:31.655102 25735 heartbeater.cc:511] Master 127.31.63.126:46749 requested a full tablet report, sending...
I20260809 08:46:31.655551 25266 ts_manager.cc:194] Registered new tserver with Master: 326bea9497ab4958b981f5456d1a6f2b (127.31.63.66:45009)
I20260809 08:46:31.655937 25266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.66:50099
I20260809 08:46:31.656687 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 25546
I20260809 08:46:31.656834 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:46:31.657034 31997 external_mini_cluster.cc:1428] Setting key 7bf082dbfa1d23ccd2a3eb933ae7e37f
I20260809 08:46:31.658138 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46749
--builtin_ntp_servers=127.31.63.84:35189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:46:31.743116 25739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:31.743317 25739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:31.743340 25739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:31.744441 25739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:31.744479 25739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:31.744503 25739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:46:31.745846 25739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:31.746119 25739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:31.746421 25739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:46:31.751775 25739 server_base.cc:1034] running on GCE node
W20260809 08:46:31.751729 25745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:31.751724 25748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:31.751737 25744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:31.752272 25739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:31.752532 25739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:31.753712 25739 hybrid_clock.cc:648] HybridClock initialized: now 1786265191753686 us; error 38 us; skew 500 ppm
I20260809 08:46:31.755127 25739 webserver.cc:460] Webserver started at http://127.31.63.67:35983/ using document root <none> and password file <none>
I20260809 08:46:31.755414 25739 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:31.755488 25739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:31.755648 25739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:31.756729 25739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "aa39abd7d7e145a4904cc22d8311ba7b"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "131e897a9818d8bc416f193634f77630"
server_key_iv: "1b850cd601cab3da5d51760e18026eb2"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.757103 25739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "aa39abd7d7e145a4904cc22d8311ba7b"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "131e897a9818d8bc416f193634f77630"
server_key_iv: "1b850cd601cab3da5d51760e18026eb2"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.758637 25739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:46:31.759545 25754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:31.759759 25739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:31.759864 25739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "aa39abd7d7e145a4904cc22d8311ba7b"
format_stamp: "Formatted at 2026-08-09 08:46:31 on dist-test-slave-gkw7"
server_key: "131e897a9818d8bc416f193634f77630"
server_key_iv: "1b850cd601cab3da5d51760e18026eb2"
server_key_version: "encryptionkey@0"
I20260809 08:46:31.759950 25739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:31.776229 25739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:31.776552 25739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:31.776710 25739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:31.776953 25739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:31.777328 25739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:31.777384 25739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:31.777420 25739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:31.777448 25739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:31.783718 25739 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:34943
I20260809 08:46:31.783779 25911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:34943 every 8 connection(s)
I20260809 08:46:31.784113 25739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:46:31.789258 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.784879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:57129 (local address 127.31.63.126:46749)
0809 08:46:31.784932 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.784935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.785210 (+ 275us) server_negotiation.cc:407] Connection header received
0809 08:46:31.785763 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.785764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.785775 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.785830 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:31.786479 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.786974 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.787921 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.788126 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.788179 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.789019 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:31.789021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:31.789021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:31.789025 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:31.789038 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:31.789042 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:31.789108 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.789174 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.789197 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:46:31.789600 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.784878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:46749 (local address 127.31.63.67:57129)
0809 08:46:31.785073 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.785107 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:31.785673 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:31.785882 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:31.785886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:31.785894 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:31.786407 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.786411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.787054 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.787055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.787845 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.787847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.788203 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.788204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.788314 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.788888 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:46:31.788888 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:31.788918 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:46:31.788940 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:31.789102 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:31.789107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:31.789109 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:46:31.789160 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 08:46:31.789239 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260809 08:46:31.790169 25912 heartbeater.cc:347] Connected to a master server at 127.31.63.126:46749
I20260809 08:46:31.790277 25912 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:31.790478 25912 heartbeater.cc:511] Master 127.31.63.126:46749 requested a full tablet report, sending...
I20260809 08:46:31.790905 25266 ts_manager.cc:194] Registered new tserver with Master: aa39abd7d7e145a4904cc22d8311ba7b (127.31.63.67:34943)
I20260809 08:46:31.791270 25266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.67:57129
I20260809 08:46:31.793991 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 25739
I20260809 08:46:31.794095 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:46:31.794303 31997 external_mini_cluster.cc:1428] Setting key 3934a350b232f2966b45331c1edd5c1a
I20260809 08:46:31.795477 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:46:31.800350 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.797238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.31.63.126:46749)
0809 08:46:31.797368 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.797370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.797496 (+ 126us) server_negotiation.cc:407] Connection header received
0809 08:46:31.797610 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.797611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.797630 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.797693 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:31.797885 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.798368 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.799150 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.799343 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.799417 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.800081 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:31.800082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:31.800083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:31.800088 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:31.800104 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:31.800112 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:31.800170 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.800233 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.800265 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 08:46:31.801963 25266 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:46:31.803817 25266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:46:31.810593 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.806273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:37239 (local address 127.0.0.1:34806)
0809 08:46:31.806393 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.806404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:31.806447 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:31.807004 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:31.807005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:31.807007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:31.807040 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.807042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.808379 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.808380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.809289 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.809291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.810233 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.810234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.810326 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.810343 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 08:46:31.810424 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 08:46:31.810498 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260809 08:46:31.810730 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.806177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.31.63.67:34943)
0809 08:46:31.806658 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.806663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.806675 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:46:31.806731 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.806733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.806747 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.806869 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:31.807077 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.807917 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.809609 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.810327 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.810420 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.810514 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.810581 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.810637 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":100,"threads_started":1}
I20260809 08:46:31.810765 25924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.806043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:34943 (local address 127.0.0.1:43972)
0809 08:46:31.806157 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.806198 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:31.806256 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:31.806873 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:31.806876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:31.806879 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:31.806956 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.806959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.808024 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.808025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.809449 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.809452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.810424 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.810425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.810499 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.810504 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:46:31.810567 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:46:31.810625 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":68,"threads_started":1}
I20260809 08:46:31.811093 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.806495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34806 (local address 127.31.63.65:37239)
0809 08:46:31.806711 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.806715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.806723 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:46:31.806765 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.806767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.806780 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.807119 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:31.807186 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.808278 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.809381 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.810122 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.810201 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.810306 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.810942 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.810988 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":105,"threads_started":1}
I20260809 08:46:31.811782 25447 tablet_service.cc:1467] Processing CreateTablet for tablet 253297944118454fa823fe11e0dea572 (DEFAULT_TABLE table=test-table [id=a71fc074cf9641d2809972c177650529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:31.811798 25816 tablet_service.cc:1467] Processing CreateTablet for tablet 253297944118454fa823fe11e0dea572 (DEFAULT_TABLE table=test-table [id=a71fc074cf9641d2809972c177650529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:31.812134 25447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 253297944118454fa823fe11e0dea572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:31.812178 25816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 253297944118454fa823fe11e0dea572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:31.814896 25930 tablet_bootstrap.cc:492] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb: Bootstrap starting.
I20260809 08:46:31.815513 25931 tablet_bootstrap.cc:492] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: Bootstrap starting.
I20260809 08:46:31.815790 25930 tablet_bootstrap.cc:654] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:31.815920 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.806168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:45009 (local address 127.0.0.1:36882)
0809 08:46:31.806387 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.806399 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:31.806461 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:31.806930 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:31.806931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:31.806933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:31.806986 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.806988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.808342 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.808343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.809592 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.809595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.810595 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.810596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.810688 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.815701 (+ 5013us) client_negotiation.cc:769] Sending connection context
0809 08:46:31.815766 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:46:31.815817 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":115,"threads_started":1}
I20260809 08:46:31.816246 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.806450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.31.63.66:45009)
0809 08:46:31.806679 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.806684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.806696 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:46:31.806742 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.806744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.806769 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.806854 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:31.807068 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.808239 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.809698 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.810343 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.810434 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.810552 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.816063 (+ 5511us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.816132 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":125,"threads_started":1}
I20260809 08:46:31.816540 25930 log.cc:824] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:31.816780 25931 tablet_bootstrap.cc:654] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:31.816979 25634 tablet_service.cc:1467] Processing CreateTablet for tablet 253297944118454fa823fe11e0dea572 (DEFAULT_TABLE table=test-table [id=a71fc074cf9641d2809972c177650529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:31.817245 25931 log.cc:824] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:31.817317 25634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 253297944118454fa823fe11e0dea572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:31.817471 25930 tablet_bootstrap.cc:492] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb: No bootstrap required, opened a new log
I20260809 08:46:31.817571 25930 ts_tablet_manager.cc:1397] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:46:31.818233 25931 tablet_bootstrap.cc:492] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: No bootstrap required, opened a new log
I20260809 08:46:31.818336 25931 ts_tablet_manager.cc:1397] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:46:31.820241 25930 raft_consensus.cc:348] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 } } peers { permanent_uuid: "d597a635a8654d53beb31067f9481ebb" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37239 } } peers { permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 } }
I20260809 08:46:31.820434 25930 raft_consensus.cc:374] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:31.820472 25930 raft_consensus.cc:729] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d597a635a8654d53beb31067f9481ebb, State: Initialized, Role: FOLLOWER
I20260809 08:46:31.820377 25931 raft_consensus.cc:348] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 } } peers { permanent_uuid: "d597a635a8654d53beb31067f9481ebb" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37239 } } peers { permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 } }
I20260809 08:46:31.820565 25931 raft_consensus.cc:374] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:31.820623 25931 raft_consensus.cc:729] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa39abd7d7e145a4904cc22d8311ba7b, State: Initialized, Role: FOLLOWER
I20260809 08:46:31.820598 25930 consensus_queue.cc:260] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 } } peers { permanent_uuid: "d597a635a8654d53beb31067f9481ebb" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37239 } } peers { permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 } }
I20260809 08:46:31.820773 25931 consensus_queue.cc:260] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 } } peers { permanent_uuid: "d597a635a8654d53beb31067f9481ebb" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37239 } } peers { permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 } }
I20260809 08:46:31.820869 25930 ts_tablet_manager.cc:1428] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:46:31.820956 25934 tablet_bootstrap.cc:492] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: Bootstrap starting.
I20260809 08:46:31.820995 25542 heartbeater.cc:503] Master 127.31.63.126:46749 was elected leader, sending a full tablet report...
I20260809 08:46:31.821148 25912 heartbeater.cc:503] Master 127.31.63.126:46749 was elected leader, sending a full tablet report...
I20260809 08:46:31.821646 25931 ts_tablet_manager.cc:1428] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:46:31.822234 25934 tablet_bootstrap.cc:654] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:31.822614 25934 log.cc:824] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:31.823421 25934 tablet_bootstrap.cc:492] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: No bootstrap required, opened a new log
I20260809 08:46:31.823487 25934 ts_tablet_manager.cc:1397] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:46:31.825899 25934 raft_consensus.cc:348] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 } } peers { permanent_uuid: "d597a635a8654d53beb31067f9481ebb" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37239 } } peers { permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 } }
I20260809 08:46:31.826045 25934 raft_consensus.cc:374] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:31.826107 25934 raft_consensus.cc:729] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326bea9497ab4958b981f5456d1a6f2b, State: Initialized, Role: FOLLOWER
I20260809 08:46:31.826249 25934 consensus_queue.cc:260] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 } } peers { permanent_uuid: "d597a635a8654d53beb31067f9481ebb" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37239 } } peers { permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 } }
I20260809 08:46:31.826606 25735 heartbeater.cc:503] Master 127.31.63.126:46749 was elected leader, sending a full tablet report...
I20260809 08:46:31.826586 25934 ts_tablet_manager.cc:1428] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:46:31.877792 25935 raft_consensus.cc:482] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:31.877954 25935 raft_consensus.cc:504] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 } } peers { permanent_uuid: "d597a635a8654d53beb31067f9481ebb" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37239 } } peers { permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 } }
I20260809 08:46:31.878382 25935 leader_election.cc:288] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa39abd7d7e145a4904cc22d8311ba7b (127.31.63.67:34943), 326bea9497ab4958b981f5456d1a6f2b (127.31.63.66:45009)
I20260809 08:46:31.882817 25944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.878627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:45009 (local address 127.31.63.65:56991)
0809 08:46:31.878856 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.878865 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:31.878914 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:31.879065 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:31.879065 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:31.879067 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:31.879136 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.879138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.880202 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.880204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.881591 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.881595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.882481 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.882482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.882560 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.882567 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:46:31.882625 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:46:31.882682 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":84,"thread_start_us":115,"threads_started":1}
I20260809 08:46:31.882817 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.878770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:42837 (local address 127.31.63.67:34943)
0809 08:46:31.878826 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.878828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.878839 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:46:31.878875 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.878876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.878888 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.879002 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:31.879218 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.880113 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.881704 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.882371 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.882478 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.882647 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.882675 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.882735 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:46:31.882860 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.878747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:56991 (local address 127.31.63.66:45009)
0809 08:46:31.878809 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.878812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.878931 (+ 119us) server_negotiation.cc:407] Connection header received
0809 08:46:31.878963 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.878965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.878980 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.879055 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:31.879217 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.880070 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.881701 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.882390 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.882478 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.882489 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.882731 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.882780 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:46:31.883132 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.878615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:34943 (local address 127.31.63.65:42837)
0809 08:46:31.878682 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.878696 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:31.878750 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:31.879001 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:31.879002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:31.879003 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:31.879113 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.879115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.880202 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.880204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.881603 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:31.881604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.882497 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.882498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:31.882560 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.882567 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:46:31.882625 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:46:31.882682 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":72}
I20260809 08:46:31.883236 25663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "253297944118454fa823fe11e0dea572" candidate_uuid: "d597a635a8654d53beb31067f9481ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326bea9497ab4958b981f5456d1a6f2b" is_pre_election: true
I20260809 08:46:31.883419 25663 raft_consensus.cc:2454] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d597a635a8654d53beb31067f9481ebb in term 0.
I20260809 08:46:31.883488 25849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "253297944118454fa823fe11e0dea572" candidate_uuid: "d597a635a8654d53beb31067f9481ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" is_pre_election: true
I20260809 08:46:31.883662 25849 raft_consensus.cc:2454] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d597a635a8654d53beb31067f9481ebb in term 0.
I20260809 08:46:31.883706 25385 leader_election.cc:302] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326bea9497ab4958b981f5456d1a6f2b, d597a635a8654d53beb31067f9481ebb; no voters:
I20260809 08:46:31.883838 25935 raft_consensus.cc:2781] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:46:31.883937 25935 raft_consensus.cc:482] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:46:31.883971 25935 raft_consensus.cc:3037] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:31.884804 25935 raft_consensus.cc:504] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 } } peers { permanent_uuid: "d597a635a8654d53beb31067f9481ebb" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37239 } } peers { permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 } }
I20260809 08:46:31.885015 25935 leader_election.cc:288] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [CANDIDATE]: Term 1 election: Requested vote from peers aa39abd7d7e145a4904cc22d8311ba7b (127.31.63.67:34943), 326bea9497ab4958b981f5456d1a6f2b (127.31.63.66:45009)
I20260809 08:46:31.885217 25849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "253297944118454fa823fe11e0dea572" candidate_uuid: "d597a635a8654d53beb31067f9481ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa39abd7d7e145a4904cc22d8311ba7b"
I20260809 08:46:31.885301 25849 raft_consensus.cc:3037] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:31.886191 25849 raft_consensus.cc:2454] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d597a635a8654d53beb31067f9481ebb in term 1.
I20260809 08:46:31.886333 25663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "253297944118454fa823fe11e0dea572" candidate_uuid: "d597a635a8654d53beb31067f9481ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326bea9497ab4958b981f5456d1a6f2b"
I20260809 08:46:31.886437 25387 leader_election.cc:302] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa39abd7d7e145a4904cc22d8311ba7b, d597a635a8654d53beb31067f9481ebb; no voters:
I20260809 08:46:31.886458 25663 raft_consensus.cc:3037] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:31.886543 25935 raft_consensus.cc:2781] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:31.886745 25935 raft_consensus.cc:686] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 1 LEADER]: Becoming Leader. State: Replica: d597a635a8654d53beb31067f9481ebb, State: Running, Role: LEADER
I20260809 08:46:31.886868 25935 consensus_queue.cc:237] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 } } peers { permanent_uuid: "d597a635a8654d53beb31067f9481ebb" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37239 } } peers { permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 } }
I20260809 08:46:31.887620 25663 raft_consensus.cc:2454] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d597a635a8654d53beb31067f9481ebb in term 1.
I20260809 08:46:31.888000 25266 catalog_manager.cc:5495] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb reported cstate change: term changed from 0 to 1, leader changed from <none> to d597a635a8654d53beb31067f9481ebb (127.31.63.65). New cstate: current_term: 1 leader_uuid: "d597a635a8654d53beb31067f9481ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d597a635a8654d53beb31067f9481ebb" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 } health_report { overall_health: UNKNOWN } } }
W20260809 08:46:31.899123 25736 tablet.cc:2369] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:46:31.906148 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.903820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34812 (local address 127.31.63.65:37239)
0809 08:46:31.903899 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:31.903902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:31.903911 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:46:31.903952 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:31.903952 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:31.903976 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:31.904057 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:46:31.904152 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.904676 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.905426 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:31.905704 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:31.905763 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:31.905872 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:46:31.905972 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:46:31.906028 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:31.906040 (+ 12us) server_negotiation.cc:299] Negotiation successful
0809 08:46:31.906057 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:46:31.908241 25663 raft_consensus.cc:1261] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b [term 1 FOLLOWER]: Refusing update from remote peer d597a635a8654d53beb31067f9481ebb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:31.908241 25849 raft_consensus.cc:1261] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b [term 1 FOLLOWER]: Refusing update from remote peer d597a635a8654d53beb31067f9481ebb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:31.908759 25945 consensus_queue.cc:1035] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "326bea9497ab4958b981f5456d1a6f2b" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 45009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:31.908898 25935 consensus_queue.cc:1035] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa39abd7d7e145a4904cc22d8311ba7b" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:31.911235 25950 mvcc.cc:205] Tried to move back new op lower bound from 7316542226052886528 to 7316542225969385472. Current Snapshot: MvccSnapshot[applied={T|T < 7316542226052886528}]
I20260809 08:46:31.912325 25953 mvcc.cc:205] Tried to move back new op lower bound from 7316542226052886528 to 7316542225969385472. Current Snapshot: MvccSnapshot[applied={T|T < 7316542226052886528}]
I20260809 08:46:31.913730 25265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:31.913842 25265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:31.914640 25265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbe8def50775444a9cd92f1ca76b773a: Sending DeleteTablet for 3 replicas of tablet 253297944118454fa823fe11e0dea572
I20260809 08:46:31.914987 25634 tablet_service.cc:1514] Processing DeleteTablet for tablet 253297944118454fa823fe11e0dea572 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:31 UTC) from {username='slave'} at 127.0.0.1:36882
I20260809 08:46:31.915009 25816 tablet_service.cc:1514] Processing DeleteTablet for tablet 253297944118454fa823fe11e0dea572 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:31 UTC) from {username='slave'} at 127.0.0.1:43972
I20260809 08:46:31.915222 25447 tablet_service.cc:1514] Processing DeleteTablet for tablet 253297944118454fa823fe11e0dea572 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:31 UTC) from {username='slave'} at 127.0.0.1:34806
I20260809 08:46:31.915249 25963 tablet_replica.cc:330] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: stopping tablet replica
I20260809 08:46:31.915374 25963 raft_consensus.cc:2229] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:31.915479 25965 tablet_replica.cc:330] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb: stopping tablet replica
I20260809 08:46:31.915490 25963 raft_consensus.cc:2258] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:31.915625 25965 raft_consensus.cc:2229] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:46:31.915727 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 25363
I20260809 08:46:31.915824 25965 raft_consensus.cc:2258] T 253297944118454fa823fe11e0dea572 P d597a635a8654d53beb31067f9481ebb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:31.915998 25963 ts_tablet_manager.cc:1905] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:31.917671 25963 ts_tablet_manager.cc:1918] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:31.917760 25963 log.cc:1196] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/253297944118454fa823fe11e0dea572
I20260809 08:46:31.918004 25963 ts_tablet_manager.cc:1939] T 253297944118454fa823fe11e0dea572 P aa39abd7d7e145a4904cc22d8311ba7b: Deleting consensus metadata
I20260809 08:46:31.921566 25962 tablet_replica.cc:330] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: stopping tablet replica
I20260809 08:46:31.921698 25962 raft_consensus.cc:2229] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:31.921761 25246 catalog_manager.cc:4841] TS aa39abd7d7e145a4904cc22d8311ba7b (127.31.63.67:34943): tablet 253297944118454fa823fe11e0dea572 (table test-table [id=a71fc074cf9641d2809972c177650529]) successfully deleted
I20260809 08:46:31.921886 25962 raft_consensus.cc:2258] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:31.922370 25962 ts_tablet_manager.cc:1905] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:31.923718 25962 ts_tablet_manager.cc:1918] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:31.923771 25962 log.cc:1196] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/253297944118454fa823fe11e0dea572
I20260809 08:46:31.923945 25962 ts_tablet_manager.cc:1939] T 253297944118454fa823fe11e0dea572 P 326bea9497ab4958b981f5456d1a6f2b: Deleting consensus metadata
I20260809 08:46:31.924431 25244 catalog_manager.cc:4841] TS 326bea9497ab4958b981f5456d1a6f2b (127.31.63.66:45009): tablet 253297944118454fa823fe11e0dea572 (table test-table [id=a71fc074cf9641d2809972c177650529]) successfully deleted
W20260809 08:46:31.928743 25246 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:37239 (error 108)
I20260809 08:46:31.928958 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 25546
I20260809 08:46:31.930599 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:31.929175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:37239 (local address 127.0.0.1:34822)
0809 08:46:31.930508 (+ 1333us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:31.930558 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:37239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1309}
W20260809 08:46:31.934660 25246 catalog_manager.cc:4584] TS d597a635a8654d53beb31067f9481ebb (127.31.63.65:37239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 253297944118454fa823fe11e0dea572: Network error: Client connection negotiation failed: client connection to 127.31.63.65:37239: connect: Connection refused (error 111)
I20260809 08:46:31.942807 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 25739
I20260809 08:46:31.965497 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 25228
2026-08-09T08:46:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (761 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:32 dist-test-slave-gkw7 krb5kdc[25972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:32 dist-test-slave-gkw7 krb5kdc[25972](info): set up 2 sockets
Aug 09 08:46:32 dist-test-slave-gkw7 krb5kdc[25972](info): commencing operation
krb5kdc: starting...
W20260809 08:46:33.217187 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:33 dist-test-slave-gkw7 krb5kdc[25972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:46:33.344424 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:38493
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:38493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:33.431792 25993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:33.432024 25993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:33.432063 25993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:33.433225 25993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:33.433288 25993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:33.433312 25993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:33.433342 25993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:33.433369 25993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:33.434902 25993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:38493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:38493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:33.435189 25993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:33.435424 25993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:33.440434 26001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:33.440502 26003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:33.440577 25998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:33.440857 25993 server_base.cc:1034] running on GCE node
I20260809 08:46:33.441092 25993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:33.441391 25993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:33.442587 25993 hybrid_clock.cc:648] HybridClock initialized: now 1786265193442535 us; error 58 us; skew 500 ppm
Aug 09 08:46:33 dist-test-slave-gkw7 krb5kdc[25972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265193, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:33.445147 25993 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:46:33.445662 25993 webserver.cc:460] Webserver started at http://127.31.63.126:41261/ using document root <none> and password file <none>
I20260809 08:46:33.445874 25993 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:33.445914 25993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:33.446003 25993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:33.447063 25993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "89bd704c483d4838b4b7cc3dafb8a337"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "373a0188d9f5a1d5cb43dd47f4b98c6e"
server_key_iv: "2311bbdf988c239a27e3733b6849dc51"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.447368 25993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "89bd704c483d4838b4b7cc3dafb8a337"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "373a0188d9f5a1d5cb43dd47f4b98c6e"
server_key_iv: "2311bbdf988c239a27e3733b6849dc51"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.448688 25993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:46:33.449422 26014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:33.449659 25993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:33.449755 25993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "89bd704c483d4838b4b7cc3dafb8a337"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "373a0188d9f5a1d5cb43dd47f4b98c6e"
server_key_iv: "2311bbdf988c239a27e3733b6849dc51"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.449817 25993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:33.465370 25993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:33.467996 25993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:33.468128 25993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:33.472620 25993 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:38493
I20260809 08:46:33.472654 26082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:38493 every 8 connection(s)
I20260809 08:46:33.473066 25993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:33.474069 26083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:33.476445 26083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337: Bootstrap starting.
I20260809 08:46:33.477072 26083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:33.477391 26083 log.cc:824] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:33.478246 26083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337: No bootstrap required, opened a new log
I20260809 08:46:33.479981 26083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bd704c483d4838b4b7cc3dafb8a337" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38493 } }
I20260809 08:46:33.480152 26083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:33.480176 26083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bd704c483d4838b4b7cc3dafb8a337, State: Initialized, Role: FOLLOWER
I20260809 08:46:33.480337 26083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bd704c483d4838b4b7cc3dafb8a337" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38493 } }
I20260809 08:46:33.480430 26083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:33.480386 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 25993
I20260809 08:46:33.480455 26083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:33.480522 26083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:33.480527 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:33.480763 31997 external_mini_cluster.cc:1428] Setting key 1d102ba2f3df8bffe169f76dde93a644
I20260809 08:46:33.481264 26083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bd704c483d4838b4b7cc3dafb8a337" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38493 } }
I20260809 08:46:33.481379 26083 leader_election.cc:302] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89bd704c483d4838b4b7cc3dafb8a337; no voters:
I20260809 08:46:33.481559 26083 leader_election.cc:288] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:33.481645 26088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:33.481822 26088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [term 1 LEADER]: Becoming Leader. State: Replica: 89bd704c483d4838b4b7cc3dafb8a337, State: Running, Role: LEADER
I20260809 08:46:33.481891 26083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:33.481920 26088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bd704c483d4838b4b7cc3dafb8a337" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38493 } }
I20260809 08:46:33.482441 26089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89bd704c483d4838b4b7cc3dafb8a337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bd704c483d4838b4b7cc3dafb8a337" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38493 } } }
I20260809 08:46:33.482542 26089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:33.483038 26091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89bd704c483d4838b4b7cc3dafb8a337. Latest consensus state: current_term: 1 leader_uuid: "89bd704c483d4838b4b7cc3dafb8a337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bd704c483d4838b4b7cc3dafb8a337" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 38493 } } }
I20260809 08:46:33.483145 26091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337 [sys.catalog]: This master's current role is: LEADER
W20260809 08:46:33.483758 26110 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:46:33.483815 26110 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:46:33.483861 26103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:33.484017 26103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:33.486057 26103 catalog_manager.cc:1347] Generated new cluster ID: a45dbf569a064f71a5d80dfbf4b21557
I20260809 08:46:33.486132 26103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:46:33 dist-test-slave-gkw7 krb5kdc[25972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:33.494728 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:33.481347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45852 (local address 127.31.63.126:38493)
0809 08:46:33.481672 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:33.481684 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:33.481750 (+ 66us) server_negotiation.cc:407] Connection header received
0809 08:46:33.481933 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:33.481941 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:33.482014 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:33.482946 (+ 932us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:33.483217 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.484831 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.485994 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.486563 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.486704 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:33.489373 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:33.489378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:33.489378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:33.489392 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:33.492203 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:33.492691 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:33.492695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:33.492696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:33.492847 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:33.493095 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:33.493096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:33.493096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:33.493448 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:33.493697 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:33.494227 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 08:46:33.494397 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":130,"threads_started":1}
I20260809 08:46:33.499488 26103 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:33.500077 26103 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:33.503592 26103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337: Generated new TSK 0
I20260809 08:46:33.503804 26103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:46:33.536509 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38493
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:33.633956 26119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:33.634160 26119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:33.634184 26119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:33.635332 26119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:33.635375 26119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:33.635401 26119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:46:33.636878 26119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:33.637120 26119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:33.637382 26119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:33.643033 26132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:33.643133 26130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:33.643033 26129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:33.643390 26119 server_base.cc:1034] running on GCE node
I20260809 08:46:33.643595 26119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:33.643805 26119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:33.644958 26119 hybrid_clock.cc:648] HybridClock initialized: now 1786265193644936 us; error 35 us; skew 500 ppm
Aug 09 08:46:33 dist-test-slave-gkw7 krb5kdc[25972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265193, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:33.647647 26119 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:46:33.648130 26119 webserver.cc:460] Webserver started at http://127.31.63.65:39025/ using document root <none> and password file <none>
I20260809 08:46:33.648401 26119 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:33.648473 26119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:33.648624 26119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:33.649748 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "62e117ef84014975b8ebd64324e26b52"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "cb910f50394093bc54453bfabdfbe1d8"
server_key_iv: "05173a5e4f3706541170366029cadc58"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.650106 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "62e117ef84014975b8ebd64324e26b52"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "cb910f50394093bc54453bfabdfbe1d8"
server_key_iv: "05173a5e4f3706541170366029cadc58"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.651489 26119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:46:33.652298 26140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:33.652540 26119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:33.652664 26119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "62e117ef84014975b8ebd64324e26b52"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "cb910f50394093bc54453bfabdfbe1d8"
server_key_iv: "05173a5e4f3706541170366029cadc58"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.652743 26119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:33.660348 26119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:33.662400 26119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:33.662564 26119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:33.662846 26119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:33.663175 26119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:33.663241 26119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:33.663280 26119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:33.663307 26119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:33.669497 26119 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:37699
I20260809 08:46:33.669582 26299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:37699 every 8 connection(s)
I20260809 08:46:33.669895 26119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:46:33.673095 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 26119
I20260809 08:46:33.673214 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:46:33.673452 31997 external_mini_cluster.cc:1428] Setting key e1bb257a136ab9967e6f11d097d1cbf2
Aug 09 08:46:33 dist-test-slave-gkw7 krb5kdc[25972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265193, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:33.680418 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:33.670657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:55055 (local address 127.31.63.126:38493)
0809 08:46:33.670726 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:33.670728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:33.670921 (+ 193us) server_negotiation.cc:407] Connection header received
0809 08:46:33.671402 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:33.671403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:33.671413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:33.671472 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:33.672370 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.672905 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.674418 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.674613 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.674678 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:33.677393 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:33.677394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:33.677394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:33.677401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:33.678919 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:33.679358 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:33.679360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:33.679361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:33.679466 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:33.679742 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:33.679743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:33.679744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:33.679863 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:33.679938 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:33.680234 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 08:46:33.680325 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:46:33.680627 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:33.670628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:38493 (local address 127.31.63.65:55055)
0809 08:46:33.670787 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:33.670830 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:33.671321 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:33.671522 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:33.671525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:33.672084 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:33.672293 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:33.672298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.673004 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.673005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:33.674327 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:33.674330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.674704 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.674706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:33.674858 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:33.675410 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:33.675420 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:33.677297 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:33.679024 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:33.679032 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:33.679034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:33.679275 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:33.679548 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:33.679549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:33.679549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:33.679664 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:33.679930 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:33.679937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:33.680080 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 08:46:33.680207 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 08:46:33.680369 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":97,"threads_started":1}
I20260809 08:46:33.681128 26300 heartbeater.cc:347] Connected to a master server at 127.31.63.126:38493
I20260809 08:46:33.681226 26300 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:33.681411 26300 heartbeater.cc:511] Master 127.31.63.126:38493 requested a full tablet report, sending...
I20260809 08:46:33.682039 26034 ts_manager.cc:194] Registered new tserver with Master: 62e117ef84014975b8ebd64324e26b52 (127.31.63.65:37699)
I20260809 08:46:33.683024 26034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:55055
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:46:33.702170 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38493
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:33.786753 26305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:33.786955 26305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:33.786980 26305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:33.788206 26305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:33.788251 26305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:33.788307 26305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:46:33.789775 26305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:33.790025 26305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:33.790268 26305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:33.795439 26316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:33.795545 26314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:33.795444 26313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:33.795734 26305 server_base.cc:1034] running on GCE node
I20260809 08:46:33.795945 26305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:33.796200 26305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:33.797410 26305 hybrid_clock.cc:648] HybridClock initialized: now 1786265193797393 us; error 27 us; skew 500 ppm
Aug 09 08:46:33 dist-test-slave-gkw7 krb5kdc[25972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265193, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:33.800292 26305 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:46:33.800854 26305 webserver.cc:460] Webserver started at http://127.31.63.66:37271/ using document root <none> and password file <none>
I20260809 08:46:33.801155 26305 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:33.801210 26305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:33.801484 26305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:33.802770 26305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "8fc5c9aeb1d94805b3a4e470a640eae4"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "32d43de64281f8d807fc26728cee2767"
server_key_iv: "53da8a50313899c81291a7f27e9804e8"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.803196 26305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "8fc5c9aeb1d94805b3a4e470a640eae4"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "32d43de64281f8d807fc26728cee2767"
server_key_iv: "53da8a50313899c81291a7f27e9804e8"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.804777 26305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:33.805653 26323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:33.805842 26305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:33.805931 26305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "8fc5c9aeb1d94805b3a4e470a640eae4"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "32d43de64281f8d807fc26728cee2767"
server_key_iv: "53da8a50313899c81291a7f27e9804e8"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.805987 26305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:33.813898 26305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:33.816084 26305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:33.816200 26305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:33.816375 26305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:33.816622 26305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:33.816661 26305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:33.816681 26305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:33.816694 26305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:33.822819 26305 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:46519
I20260809 08:46:33.822909 26485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:46519 every 8 connection(s)
I20260809 08:46:33.823230 26305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:46:33.827916 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 26305
I20260809 08:46:33.828037 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:46:33.828287 31997 external_mini_cluster.cc:1428] Setting key 18fe17cc68abd2f22dd60c58a6c40d4d
Aug 09 08:46:33 dist-test-slave-gkw7 krb5kdc[25972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265193, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:33.833199 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:33.823932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:40731 (local address 127.31.63.126:38493)
0809 08:46:33.823975 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:33.823978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:33.824192 (+ 214us) server_negotiation.cc:407] Connection header received
0809 08:46:33.824736 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:33.824738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:33.824752 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:33.824817 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:33.825854 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.826364 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.827354 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.827558 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.827645 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:33.830277 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:33.830279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:33.830279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:33.830288 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:33.831778 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:33.832209 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:33.832210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:33.832211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:33.832273 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:33.832535 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:33.832536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:33.832537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:33.832654 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:33.832725 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:33.833005 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 08:46:33.833100 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:46:33.833477 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:33.823904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:38493 (local address 127.31.63.66:40731)
0809 08:46:33.824064 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:33.824100 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:33.824629 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:33.824888 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:33.824892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:33.825487 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:33.825758 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:33.825763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.826460 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.826461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:33.827260 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:33.827263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.827653 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.827654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:33.827797 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:33.828318 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:33.828344 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:33.830178 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:33.831884 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:33.831892 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:33.831894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:33.832128 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:33.832345 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:33.832346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:33.832346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:33.832459 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:33.832728 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:33.832736 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:33.832874 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 08:46:33.832984 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 08:46:33.833175 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260809 08:46:33.834022 26486 heartbeater.cc:347] Connected to a master server at 127.31.63.126:38493
I20260809 08:46:33.834131 26486 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:33.834316 26486 heartbeater.cc:511] Master 127.31.63.126:38493 requested a full tablet report, sending...
I20260809 08:46:33.834741 26034 ts_manager.cc:194] Registered new tserver with Master: 8fc5c9aeb1d94805b3a4e470a640eae4 (127.31.63.66:46519)
I20260809 08:46:33.835296 26034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:40731
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:46:33.854678 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38493
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:33.939733 26493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:33.939937 26493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:33.939960 26493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:33.941035 26493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:33.941078 26493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:33.941105 26493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:46:33.942620 26493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:38493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:33.942814 26493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:33.942987 26493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:33.948254 26501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:33.948261 26505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:33.948261 26500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:33.948534 26493 server_base.cc:1034] running on GCE node
I20260809 08:46:33.948730 26493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:33.948925 26493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:33.950122 26493 hybrid_clock.cc:648] HybridClock initialized: now 1786265193950103 us; error 30 us; skew 500 ppm
Aug 09 08:46:33 dist-test-slave-gkw7 krb5kdc[25972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265193, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:33.952639 26493 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:46:33.953104 26493 webserver.cc:460] Webserver started at http://127.31.63.67:43453/ using document root <none> and password file <none>
I20260809 08:46:33.953291 26493 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:33.953332 26493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:33.953507 26493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:33.954578 26493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "f03157967f1e4d07aca3adeeb42edeea"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "be38c26f049a7693ae13835470df0b43"
server_key_iv: "c63f734fb2f5c186e07ac311146d3885"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.954902 26493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "f03157967f1e4d07aca3adeeb42edeea"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "be38c26f049a7693ae13835470df0b43"
server_key_iv: "c63f734fb2f5c186e07ac311146d3885"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.956363 26493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:46:33.957067 26517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:33.957260 26493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:46:33.957332 26493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "f03157967f1e4d07aca3adeeb42edeea"
format_stamp: "Formatted at 2026-08-09 08:46:33 on dist-test-slave-gkw7"
server_key: "be38c26f049a7693ae13835470df0b43"
server_key_iv: "c63f734fb2f5c186e07ac311146d3885"
server_key_version: "encryptionkey@0"
I20260809 08:46:33.957382 26493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:33.967440 26493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:33.969179 26493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:33.969321 26493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:33.969681 26493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:33.970012 26493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:33.970047 26493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:33.970108 26493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:33.970130 26493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:33.977140 26493 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:41821
I20260809 08:46:33.977223 26677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:41821 every 8 connection(s)
I20260809 08:46:33.977631 26493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:46:33.980413 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 26493
I20260809 08:46:33.980553 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:46:33.980731 31997 external_mini_cluster.cc:1428] Setting key 9412e8452eb05cb98439a97e5af52169
Aug 09 08:46:33 dist-test-slave-gkw7 krb5kdc[25972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265193, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:33.988344 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:33.978499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:49203 (local address 127.31.63.126:38493)
0809 08:46:33.978561 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:33.978564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:33.978754 (+ 190us) server_negotiation.cc:407] Connection header received
0809 08:46:33.979256 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:33.979258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:33.979271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:33.979341 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:33.980359 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.981061 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.982443 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.982779 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.982860 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:33.985358 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:33.985360 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:33.985361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:33.985369 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:33.986801 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:33.987269 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:33.987270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:33.987271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:33.987339 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:33.987629 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:33.987630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:33.987631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:33.987763 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:33.987845 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:33.988152 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 08:46:33.988239 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:46:33.988574 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:33.978439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:38493 (local address 127.31.63.67:49203)
0809 08:46:33.978606 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:33.978648 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:33.979133 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:33.979408 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:33.979412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:33.980004 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:33.980244 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:33.980251 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.981174 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.981176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:33.982296 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:33.982299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.982891 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.982892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:33.983007 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:33.983655 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:33.983665 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:33.985248 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:33.986896 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:33.986904 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:33.986907 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:33.987168 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:33.987425 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:33.987426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:33.987427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:33.987550 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:33.987851 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:33.987859 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:33.988007 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 08:46:33.988127 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 08:46:33.988306 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260809 08:46:33.989105 26678 heartbeater.cc:347] Connected to a master server at 127.31.63.126:38493
I20260809 08:46:33.989220 26678 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:33.989477 26678 heartbeater.cc:511] Master 127.31.63.126:38493 requested a full tablet report, sending...
I20260809 08:46:33.989967 26034 ts_manager.cc:194] Registered new tserver with Master: f03157967f1e4d07aca3adeeb42edeea (127.31.63.67:41821)
I20260809 08:46:33.990353 26034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:49203
I20260809 08:46:33.992587 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:46:34.001121 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:33.994300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45862 (local address 127.31.63.126:38493)
0809 08:46:33.994350 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:33.994352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:33.994472 (+ 120us) server_negotiation.cc:407] Connection header received
0809 08:46:33.994577 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:33.994578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:33.994589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:33.994641 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:33.995332 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.995825 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.996674 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:33.996858 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:33.996917 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:33.998460 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:33.998462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:33.998462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:33.998468 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:33.999866 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:34.000275 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:34.000276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:34.000277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:34.000335 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:34.000545 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:34.000546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:34.000546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:34.000699 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:34.000785 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.000929 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.001014 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:46:34.002835 26034 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:46:34.004565 26034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:46:34.011586 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.006976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:46519 (local address 127.0.0.1:35684)
0809 08:46:34.007293 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.007335 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.007404 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.007877 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.007878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.007880 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:34.007955 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.007958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.009486 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.009487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.010538 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.010541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.011248 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.011250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.011356 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.011367 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.011451 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.011508 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":46,"thread_start_us":76,"threads_started":1}
I20260809 08:46:34.011595 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.007143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.31.63.66:46519)
0809 08:46:34.007680 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.007687 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.007699 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:46:34.007742 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.007744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.007759 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.008048 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:34.008152 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.009348 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.010636 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.011145 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.011238 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.011311 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.011468 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.011511 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":102,"threads_started":1}
I20260809 08:46:34.012096 26697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.007171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:37699 (local address 127.0.0.1:36502)
0809 08:46:34.007498 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.007509 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.007565 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.007851 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.007852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.007855 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:34.007955 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.007959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.009486 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.009487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.010785 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.010789 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.011834 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.011835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.011900 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.011904 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.011956 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.012003 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":119,"thread_start_us":62,"threads_started":1}
I20260809 08:46:34.012178 26699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.007158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.31.63.67:41821)
0809 08:46:34.007567 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.007572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.007582 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:46:34.007624 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.007626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.007642 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.007717 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:34.007994 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.008857 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.010278 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.011136 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.011245 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.011308 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.012052 (+ 744us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.012102 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":106,"threads_started":1}
I20260809 08:46:34.012200 26696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.006976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:41821 (local address 127.0.0.1:46740)
0809 08:46:34.007359 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.007371 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.007468 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.007841 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.007844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.007846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:34.007926 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.007929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.008960 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.008961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.010180 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.010183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.011806 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.011807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.011870 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.011874 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.011925 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.011968 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":3,"thread_start_us":53,"threads_started":1}
I20260809 08:46:34.012471 26702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.007307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36502 (local address 127.31.63.65:37699)
0809 08:46:34.007514 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.007520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.007557 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:46:34.007638 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.007640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.007659 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.007750 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:34.008055 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.009348 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.010900 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.011621 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.011720 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.011790 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.012298 (+ 508us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.012352 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":90,"threads_started":1}
I20260809 08:46:34.012363 26390 tablet_service.cc:1467] Processing CreateTablet for tablet 494e62fd99d14b5b93076ccd05e692f7 (DEFAULT_TABLE table=test-table [id=32221bec0d7246d2a436e198db3dc9b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:34.012744 26390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494e62fd99d14b5b93076ccd05e692f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:34.015623 26703 tablet_bootstrap.cc:492] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: Bootstrap starting.
I20260809 08:46:34.016368 26205 tablet_service.cc:1467] Processing CreateTablet for tablet 494e62fd99d14b5b93076ccd05e692f7 (DEFAULT_TABLE table=test-table [id=32221bec0d7246d2a436e198db3dc9b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:34.016582 26703 tablet_bootstrap.cc:654] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:34.016772 26205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494e62fd99d14b5b93076ccd05e692f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:34.017289 26703 log.cc:824] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:34.018006 26587 tablet_service.cc:1467] Processing CreateTablet for tablet 494e62fd99d14b5b93076ccd05e692f7 (DEFAULT_TABLE table=test-table [id=32221bec0d7246d2a436e198db3dc9b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:34.018244 26703 tablet_bootstrap.cc:492] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: No bootstrap required, opened a new log
I20260809 08:46:34.018343 26703 ts_tablet_manager.cc:1397] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 08:46:34.018352 26587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 494e62fd99d14b5b93076ccd05e692f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:34.020227 26705 tablet_bootstrap.cc:492] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52: Bootstrap starting.
I20260809 08:46:34.020445 26703 raft_consensus.cc:348] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 } } peers { permanent_uuid: "f03157967f1e4d07aca3adeeb42edeea" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41821 } } peers { permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 } }
I20260809 08:46:34.020655 26703 raft_consensus.cc:374] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:34.020699 26703 raft_consensus.cc:729] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fc5c9aeb1d94805b3a4e470a640eae4, State: Initialized, Role: FOLLOWER
I20260809 08:46:34.020830 26703 consensus_queue.cc:260] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 } } peers { permanent_uuid: "f03157967f1e4d07aca3adeeb42edeea" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41821 } } peers { permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 } }
I20260809 08:46:34.021067 26705 tablet_bootstrap.cc:654] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:34.021139 26703 ts_tablet_manager.cc:1428] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:46:34.021296 26486 heartbeater.cc:503] Master 127.31.63.126:38493 was elected leader, sending a full tablet report...
I20260809 08:46:34.021889 26707 tablet_bootstrap.cc:492] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: Bootstrap starting.
I20260809 08:46:34.021834 26705 log.cc:824] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:34.022549 26707 tablet_bootstrap.cc:654] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:34.023108 26707 log.cc:824] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:34.023216 26705 tablet_bootstrap.cc:492] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52: No bootstrap required, opened a new log
I20260809 08:46:34.023299 26705 ts_tablet_manager.cc:1397] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:46:34.023813 26707 tablet_bootstrap.cc:492] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: No bootstrap required, opened a new log
I20260809 08:46:34.023869 26707 ts_tablet_manager.cc:1397] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:46:34.024930 26705 raft_consensus.cc:348] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 } } peers { permanent_uuid: "f03157967f1e4d07aca3adeeb42edeea" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41821 } } peers { permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 } }
I20260809 08:46:34.025120 26705 raft_consensus.cc:374] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:34.025174 26705 raft_consensus.cc:729] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e117ef84014975b8ebd64324e26b52, State: Initialized, Role: FOLLOWER
I20260809 08:46:34.025303 26705 consensus_queue.cc:260] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 } } peers { permanent_uuid: "f03157967f1e4d07aca3adeeb42edeea" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41821 } } peers { permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 } }
I20260809 08:46:34.025318 26707 raft_consensus.cc:348] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 } } peers { permanent_uuid: "f03157967f1e4d07aca3adeeb42edeea" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41821 } } peers { permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 } }
I20260809 08:46:34.025477 26707 raft_consensus.cc:374] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:34.025545 26707 raft_consensus.cc:729] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f03157967f1e4d07aca3adeeb42edeea, State: Initialized, Role: FOLLOWER
I20260809 08:46:34.025663 26705 ts_tablet_manager.cc:1428] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:46:34.025700 26300 heartbeater.cc:503] Master 127.31.63.126:38493 was elected leader, sending a full tablet report...
I20260809 08:46:34.025677 26707 consensus_queue.cc:260] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 } } peers { permanent_uuid: "f03157967f1e4d07aca3adeeb42edeea" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41821 } } peers { permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 } }
I20260809 08:46:34.025972 26707 ts_tablet_manager.cc:1428] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:46:34.026018 26678 heartbeater.cc:503] Master 127.31.63.126:38493 was elected leader, sending a full tablet report...
W20260809 08:46:34.073839 26488 tablet.cc:2369] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:46:34.161295 26713 raft_consensus.cc:482] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:34.161433 26713 raft_consensus.cc:504] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 } } peers { permanent_uuid: "f03157967f1e4d07aca3adeeb42edeea" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41821 } } peers { permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 } }
I20260809 08:46:34.161860 26713 leader_election.cc:288] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fc5c9aeb1d94805b3a4e470a640eae4 (127.31.63.66:46519), 62e117ef84014975b8ebd64324e26b52 (127.31.63.65:37699)
I20260809 08:46:34.166465 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.162080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:37699 (local address 127.31.63.67:57323)
0809 08:46:34.162139 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.162152 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.162198 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.162416 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.162419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.162426 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:34.162531 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.162533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.163608 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.163609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.165036 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.165040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.166172 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.166173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.166256 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.166260 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.166321 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.166368 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":107}
I20260809 08:46:34.166509 26702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.162143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:57323 (local address 127.31.63.65:37699)
0809 08:46:34.162187 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.162189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.162200 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:46:34.162298 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.162299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.162310 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.162404 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:34.162627 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.163492 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.165616 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.166072 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.166174 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.166185 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.166358 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.166440 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:46:34.166808 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.162093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:46519 (local address 127.31.63.67:54761)
0809 08:46:34.162270 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.162282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.162335 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.162474 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.162475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.162476 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:34.162532 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.162533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.163595 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.163597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.165036 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.165040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.166471 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.166473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.166559 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.166564 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.166693 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.166734 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":74,"thread_start_us":126,"threads_started":1}
I20260809 08:46:34.166821 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.162164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:54761 (local address 127.31.63.66:46519)
0809 08:46:34.162218 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.162220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.162354 (+ 134us) server_negotiation.cc:407] Connection header received
0809 08:46:34.162385 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.162386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.162397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.162460 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:34.162626 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.163465 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.165189 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.166374 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.166443 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.166452 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.166705 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.166754 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:46:34.166942 26236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494e62fd99d14b5b93076ccd05e692f7" candidate_uuid: "f03157967f1e4d07aca3adeeb42edeea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e117ef84014975b8ebd64324e26b52" is_pre_election: true
I20260809 08:46:34.167128 26236 raft_consensus.cc:2454] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f03157967f1e4d07aca3adeeb42edeea in term 0.
I20260809 08:46:34.167232 26417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494e62fd99d14b5b93076ccd05e692f7" candidate_uuid: "f03157967f1e4d07aca3adeeb42edeea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" is_pre_election: true
I20260809 08:46:34.167361 26417 raft_consensus.cc:2454] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f03157967f1e4d07aca3adeeb42edeea in term 0.
I20260809 08:46:34.167548 26519 leader_election.cc:302] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fc5c9aeb1d94805b3a4e470a640eae4, f03157967f1e4d07aca3adeeb42edeea; no voters:
I20260809 08:46:34.167723 26713 raft_consensus.cc:2781] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:46:34.167780 26713 raft_consensus.cc:482] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:46:34.167797 26713 raft_consensus.cc:3037] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:34.168498 26713 raft_consensus.cc:504] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 } } peers { permanent_uuid: "f03157967f1e4d07aca3adeeb42edeea" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41821 } } peers { permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 } }
I20260809 08:46:34.168638 26713 leader_election.cc:288] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [CANDIDATE]: Term 1 election: Requested vote from peers 8fc5c9aeb1d94805b3a4e470a640eae4 (127.31.63.66:46519), 62e117ef84014975b8ebd64324e26b52 (127.31.63.65:37699)
I20260809 08:46:34.168798 26417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494e62fd99d14b5b93076ccd05e692f7" candidate_uuid: "f03157967f1e4d07aca3adeeb42edeea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4"
I20260809 08:46:34.168805 26236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "494e62fd99d14b5b93076ccd05e692f7" candidate_uuid: "f03157967f1e4d07aca3adeeb42edeea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e117ef84014975b8ebd64324e26b52"
I20260809 08:46:34.168883 26236 raft_consensus.cc:3037] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:34.168885 26417 raft_consensus.cc:3037] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:34.169890 26417 raft_consensus.cc:2454] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f03157967f1e4d07aca3adeeb42edeea in term 1.
I20260809 08:46:34.169890 26236 raft_consensus.cc:2454] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f03157967f1e4d07aca3adeeb42edeea in term 1.
I20260809 08:46:34.170086 26519 leader_election.cc:302] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fc5c9aeb1d94805b3a4e470a640eae4, f03157967f1e4d07aca3adeeb42edeea; no voters:
I20260809 08:46:34.170195 26713 raft_consensus.cc:2781] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:34.170410 26713 raft_consensus.cc:686] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 1 LEADER]: Becoming Leader. State: Replica: f03157967f1e4d07aca3adeeb42edeea, State: Running, Role: LEADER
I20260809 08:46:34.170516 26713 consensus_queue.cc:237] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 } } peers { permanent_uuid: "f03157967f1e4d07aca3adeeb42edeea" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41821 } } peers { permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 } }
W20260809 08:46:34.170702 26301 tablet.cc:2369] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:46:34.171468 26034 catalog_manager.cc:5495] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea reported cstate change: term changed from 0 to 1, leader changed from <none> to f03157967f1e4d07aca3adeeb42edeea (127.31.63.67). New cstate: current_term: 1 leader_uuid: "f03157967f1e4d07aca3adeeb42edeea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f03157967f1e4d07aca3adeeb42edeea" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 41821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 } health_report { overall_health: UNKNOWN } } }
I20260809 08:46:34.193943 26699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.191281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.31.63.67:41821)
0809 08:46:34.191348 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.191350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.191359 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:46:34.191414 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.191415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.191433 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.191495 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:46:34.191641 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.192182 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.192986 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.193261 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.193348 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.193477 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:46:34.193602 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:46:34.193674 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.193818 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.193833 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:46:34.195847 26417 raft_consensus.cc:1261] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4 [term 1 FOLLOWER]: Refusing update from remote peer f03157967f1e4d07aca3adeeb42edeea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:34.195849 26236 raft_consensus.cc:1261] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52 [term 1 FOLLOWER]: Refusing update from remote peer f03157967f1e4d07aca3adeeb42edeea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:34.196245 26715 consensus_queue.cc:1035] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fc5c9aeb1d94805b3a4e470a640eae4" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:34.196434 26713 consensus_queue.cc:1035] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e117ef84014975b8ebd64324e26b52" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 37699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:34.199092 26722 mvcc.cc:205] Tried to move back new op lower bound from 7316542235423444992 to 7316542235323232256. Current Snapshot: MvccSnapshot[applied={T|T < 7316542235423444992}]
I20260809 08:46:34.199216 26721 mvcc.cc:205] Tried to move back new op lower bound from 7316542235423444992 to 7316542235323232256. Current Snapshot: MvccSnapshot[applied={T|T < 7316542235423444992}]
I20260809 08:46:34.201515 26033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:34.201645 26033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:34.202467 26033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89bd704c483d4838b4b7cc3dafb8a337: Sending DeleteTablet for 3 replicas of tablet 494e62fd99d14b5b93076ccd05e692f7
I20260809 08:46:34.202839 26587 tablet_service.cc:1514] Processing DeleteTablet for tablet 494e62fd99d14b5b93076ccd05e692f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:34 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:46740
I20260809 08:46:34.202841 26205 tablet_service.cc:1514] Processing DeleteTablet for tablet 494e62fd99d14b5b93076ccd05e692f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:34 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:36502
I20260809 08:46:34.203090 26733 tablet_replica.cc:330] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: stopping tablet replica
I20260809 08:46:34.203088 26390 tablet_service.cc:1514] Processing DeleteTablet for tablet 494e62fd99d14b5b93076ccd05e692f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:34 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:35684
I20260809 08:46:34.203233 26733 raft_consensus.cc:2229] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:46:34.203351 26733 raft_consensus.cc:2258] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:34.203354 26734 tablet_replica.cc:330] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: stopping tablet replica
I20260809 08:46:34.203502 26731 tablet_replica.cc:330] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52: stopping tablet replica
I20260809 08:46:34.203534 26734 raft_consensus.cc:2229] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:34.203604 26731 raft_consensus.cc:2229] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:34.203706 26731 raft_consensus.cc:2258] T 494e62fd99d14b5b93076ccd05e692f7 P 62e117ef84014975b8ebd64324e26b52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:34.203698 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 26119
I20260809 08:46:34.203794 26734 raft_consensus.cc:2258] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:34.204134 26733 ts_tablet_manager.cc:1905] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:34.204370 26734 ts_tablet_manager.cc:1905] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:34.206190 26734 ts_tablet_manager.cc:1918] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:34.206336 26734 log.cc:1196] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/494e62fd99d14b5b93076ccd05e692f7
I20260809 08:46:34.206447 26733 ts_tablet_manager.cc:1918] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:34.206532 26733 log.cc:1196] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/494e62fd99d14b5b93076ccd05e692f7
I20260809 08:46:34.206821 26733 ts_tablet_manager.cc:1939] T 494e62fd99d14b5b93076ccd05e692f7 P f03157967f1e4d07aca3adeeb42edeea: Deleting consensus metadata
I20260809 08:46:34.206940 26734 ts_tablet_manager.cc:1939] T 494e62fd99d14b5b93076ccd05e692f7 P 8fc5c9aeb1d94805b3a4e470a640eae4: Deleting consensus metadata
I20260809 08:46:34.207235 26019 catalog_manager.cc:4841] TS f03157967f1e4d07aca3adeeb42edeea (127.31.63.67:41821): tablet 494e62fd99d14b5b93076ccd05e692f7 (table test-table [id=32221bec0d7246d2a436e198db3dc9b9]) successfully deleted
I20260809 08:46:34.207683 26016 catalog_manager.cc:4841] TS 8fc5c9aeb1d94805b3a4e470a640eae4 (127.31.63.66:46519): tablet 494e62fd99d14b5b93076ccd05e692f7 (table test-table [id=32221bec0d7246d2a436e198db3dc9b9]) successfully deleted
W20260809 08:46:34.217006 26019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:37699 (error 108)
I20260809 08:46:34.217235 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 26305
I20260809 08:46:34.225790 26696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.225623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:37699 (local address 127.0.0.1:36516)
0809 08:46:34.225692 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.225746 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 08:46:34.225934 26019 catalog_manager.cc:4584] TS 62e117ef84014975b8ebd64324e26b52 (127.31.63.65:37699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 494e62fd99d14b5b93076ccd05e692f7: Network error: Client connection negotiation failed: client connection to 127.31.63.65:37699: connect: Connection refused (error 111)
I20260809 08:46:34.231339 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 26493
I20260809 08:46:34.240295 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 25993
2026-08-09T08:46:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2273 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T08:46:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:34Z Disabled control of system clock
I20260809 08:46:34.275135 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33115
--webserver_interface=127.31.63.126
--webserver_port=0
--builtin_ntp_servers=127.31.63.84:40197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:46:34.358297 26740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:34.358515 26740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:34.358537 26740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:34.359629 26740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:34.359663 26740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:34.359676 26740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:34.359689 26740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:34.359701 26740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:34.360980 26740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:34.361166 26740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:34.361397 26740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:34.366357 26747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:34.366451 26740 server_base.cc:1034] running on GCE node
W20260809 08:46:34.366330 26756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:34.366365 26748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:34.366780 26740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:34.367107 26740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:34.368328 26740 hybrid_clock.cc:648] HybridClock initialized: now 1786265194368310 us; error 46 us; skew 500 ppm
I20260809 08:46:34.369733 26740 webserver.cc:460] Webserver started at http://127.31.63.126:41909/ using document root <none> and password file <none>
I20260809 08:46:34.370011 26740 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:34.370095 26740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:34.370258 26740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:34.371397 26740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "b6460e150a0d49329ebdf2ce51edf1d0"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "ed635393ff6483675785e2d74060e9a3"
server_key_iv: "666d5e50bc8fb218724cff68a0f8f16e"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.371778 26740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "b6460e150a0d49329ebdf2ce51edf1d0"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "ed635393ff6483675785e2d74060e9a3"
server_key_iv: "666d5e50bc8fb218724cff68a0f8f16e"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.373235 26740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:46:34.374151 26762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.374377 26740 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:46:34.374500 26740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "b6460e150a0d49329ebdf2ce51edf1d0"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "ed635393ff6483675785e2d74060e9a3"
server_key_iv: "666d5e50bc8fb218724cff68a0f8f16e"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.374585 26740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:34.388245 26740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:34.388581 26740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:34.388746 26740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:34.393121 26740 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:33115
I20260809 08:46:34.393157 26844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:33115 every 8 connection(s)
I20260809 08:46:34.393517 26740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:34.394630 26845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:34.397130 26845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0: Bootstrap starting.
I20260809 08:46:34.397845 26845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:34.398164 26845 log.cc:824] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:34.398959 26845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0: No bootstrap required, opened a new log
I20260809 08:46:34.399892 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 26740
I20260809 08:46:34.399981 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:34.400188 31997 external_mini_cluster.cc:1428] Setting key c74979b9d54ea94d7dafc8fd6a4ac389
I20260809 08:46:34.400906 26845 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6460e150a0d49329ebdf2ce51edf1d0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33115 } }
I20260809 08:46:34.401094 26845 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:34.401140 26845 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6460e150a0d49329ebdf2ce51edf1d0, State: Initialized, Role: FOLLOWER
I20260809 08:46:34.401316 26845 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6460e150a0d49329ebdf2ce51edf1d0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33115 } }
I20260809 08:46:34.401415 26845 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:34.401455 26845 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:34.401499 26845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:34.402339 26845 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6460e150a0d49329ebdf2ce51edf1d0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33115 } }
I20260809 08:46:34.402452 26845 leader_election.cc:302] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6460e150a0d49329ebdf2ce51edf1d0; no voters:
I20260809 08:46:34.402626 26845 leader_election.cc:288] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:34.402702 26850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:34.402849 26850 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [term 1 LEADER]: Becoming Leader. State: Replica: b6460e150a0d49329ebdf2ce51edf1d0, State: Running, Role: LEADER
I20260809 08:46:34.402874 26845 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:34.402961 26850 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6460e150a0d49329ebdf2ce51edf1d0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33115 } }
I20260809 08:46:34.403297 26853 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6460e150a0d49329ebdf2ce51edf1d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6460e150a0d49329ebdf2ce51edf1d0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33115 } } }
I20260809 08:46:34.403374 26853 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:34.403611 26856 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6460e150a0d49329ebdf2ce51edf1d0. Latest consensus state: current_term: 1 leader_uuid: "b6460e150a0d49329ebdf2ce51edf1d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6460e150a0d49329ebdf2ce51edf1d0" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33115 } } }
I20260809 08:46:34.403743 26856 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:34.403996 26864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:34.404639 26864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:34.406167 26864 catalog_manager.cc:1347] Generated new cluster ID: c9fbf63d712346519cfaa254d707b33c
I20260809 08:46:34.406234 26864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:34.406949 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.400797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.31.63.126:33115)
0809 08:46:34.401042 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.401050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.401075 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:46:34.401198 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.401206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.401277 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.401425 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:34.401668 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.403529 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.404747 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.405370 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.405491 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.406238 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:34.406241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:34.406242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:34.406447 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:34.406487 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:34.406501 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:34.406567 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.406672 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.406753 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":98,"threads_started":1}
I20260809 08:46:34.431936 26864 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:34.432701 26864 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:34.442788 26864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0: Generated new TSK 0
I20260809 08:46:34.443009 26864 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:34.454303 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33115
--builtin_ntp_servers=127.31.63.84:40197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:46:34.540053 26879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:34.540252 26879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:34.540274 26879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:34.541334 26879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:34.541366 26879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:34.541390 26879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:46:34.542783 26879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:34.542990 26879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:34.543237 26879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:34.548436 26889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:34.548452 26887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:34.548452 26885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:34.548739 26879 server_base.cc:1034] running on GCE node
I20260809 08:46:34.548949 26879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:34.549157 26879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:34.550340 26879 hybrid_clock.cc:648] HybridClock initialized: now 1786265194550318 us; error 30 us; skew 500 ppm
I20260809 08:46:34.551651 26879 webserver.cc:460] Webserver started at http://127.31.63.65:36417/ using document root <none> and password file <none>
I20260809 08:46:34.551918 26879 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:34.552006 26879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:34.552172 26879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:34.553304 26879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "acba4c67ece747f0b5acd876d153257c"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "19c7735866ff3da1bde7f4b8d529b7dd"
server_key_iv: "ff463365bdd4e1f1e80670f508d6c4c5"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.553714 26879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "acba4c67ece747f0b5acd876d153257c"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "19c7735866ff3da1bde7f4b8d529b7dd"
server_key_iv: "ff463365bdd4e1f1e80670f508d6c4c5"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.555056 26879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:34.555856 26898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.556059 26879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:34.556162 26879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "acba4c67ece747f0b5acd876d153257c"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "19c7735866ff3da1bde7f4b8d529b7dd"
server_key_iv: "ff463365bdd4e1f1e80670f508d6c4c5"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.556246 26879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:34.570468 26879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:34.570780 26879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:34.570935 26879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:34.571165 26879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:34.571512 26879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:34.571571 26879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.571619 26879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:34.571647 26879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.577979 26879 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:35239
I20260809 08:46:34.578080 27054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:35239 every 8 connection(s)
I20260809 08:46:34.578393 26879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:46:34.580142 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 26879
I20260809 08:46:34.580265 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:46:34.580456 31997 external_mini_cluster.cc:1428] Setting key 33ed59724cd5178b97cdde92ff039df7
I20260809 08:46:34.581730 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33115
--builtin_ntp_servers=127.31.63.84:40197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 08:46:34.584901 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.579204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:48305 (local address 127.31.63.126:33115)
0809 08:46:34.579278 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.579280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.579542 (+ 262us) server_negotiation.cc:407] Connection header received
0809 08:46:34.580210 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.580212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.580230 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.580287 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:34.581227 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.582047 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.583105 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.583435 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.583518 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.584602 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:34.584604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:34.584604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:34.584608 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:34.584628 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:34.584633 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:34.584688 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.584806 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.584839 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 08:46:34.585192 27060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.579195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33115 (local address 127.31.63.65:48305)
0809 08:46:34.579401 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.579449 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.580052 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.580369 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.580372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.580381 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:34.581116 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.581123 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.582136 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.582138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.583009 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.583012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.583546 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.583548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.583760 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.584417 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:46:34.584418 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:34.584461 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:46:34.584493 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:34.584706 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:34.584712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:34.584716 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.584795 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.584916 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260809 08:46:34.585839 27057 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33115
I20260809 08:46:34.585932 27057 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:34.586127 27057 heartbeater.cc:511] Master 127.31.63.126:33115 requested a full tablet report, sending...
I20260809 08:46:34.586771 26785 ts_manager.cc:194] Registered new tserver with Master: acba4c67ece747f0b5acd876d153257c (127.31.63.65:35239)
I20260809 08:46:34.587859 26785 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.65:48305
W20260809 08:46:34.670343 27061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:34.670550 27061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:34.670572 27061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:34.671659 27061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:34.671694 27061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:34.671718 27061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:46:34.673046 27061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:34.673244 27061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:34.673504 27061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:34.678717 27073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:34.678776 27068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:34.678771 27070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:34.679060 27061 server_base.cc:1034] running on GCE node
I20260809 08:46:34.679411 27061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:34.679701 27061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:34.680943 27061 hybrid_clock.cc:648] HybridClock initialized: now 1786265194680925 us; error 33 us; skew 500 ppm
I20260809 08:46:34.682386 27061 webserver.cc:460] Webserver started at http://127.31.63.66:44379/ using document root <none> and password file <none>
I20260809 08:46:34.682669 27061 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:34.682745 27061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:34.682899 27061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:34.683995 27061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "58143f7bce4843a69ef0360b899d5ac3"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "10a6e904c67b75b98b29e01c56d26b05"
server_key_iv: "bf9a5c8271108d76acc30c0e9e0519f8"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.684377 27061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "58143f7bce4843a69ef0360b899d5ac3"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "10a6e904c67b75b98b29e01c56d26b05"
server_key_iv: "bf9a5c8271108d76acc30c0e9e0519f8"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.685776 27061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:34.686766 27083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.686956 27061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:34.687090 27061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "58143f7bce4843a69ef0360b899d5ac3"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "10a6e904c67b75b98b29e01c56d26b05"
server_key_iv: "bf9a5c8271108d76acc30c0e9e0519f8"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.687170 27061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:34.698311 27061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:34.698627 27061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:34.698778 27061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:34.699020 27061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:34.699359 27061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:34.699416 27061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.699458 27061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:34.699484 27061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.705904 27061 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:35381
I20260809 08:46:34.705979 27247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:35381 every 8 connection(s)
I20260809 08:46:34.706327 27061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:46:34.708540 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 27061
I20260809 08:46:34.708691 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:46:34.708887 31997 external_mini_cluster.cc:1428] Setting key 3a8cc32eec515f93a103ca367cf8412f
I20260809 08:46:34.710078 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33115
--builtin_ntp_servers=127.31.63.84:40197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 08:46:34.713126 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.707319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:40641 (local address 127.31.63.126:33115)
0809 08:46:34.707410 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.707412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.707695 (+ 283us) server_negotiation.cc:407] Connection header received
0809 08:46:34.708217 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.708218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.708230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.708296 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:34.709184 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.709852 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.711466 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.711663 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.711721 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.712800 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:34.712802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:34.712802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:34.712808 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:34.712827 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:34.712832 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:34.712893 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.712986 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.713016 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 08:46:34.713385 27250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.707295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33115 (local address 127.31.63.66:40641)
0809 08:46:34.707529 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.707574 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.708109 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.708381 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.708385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.708394 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:34.709085 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.709091 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.709954 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.709956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.711367 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.711372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.711760 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.711762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.711952 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.712633 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:46:34.712634 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:34.712680 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:46:34.712705 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:34.712901 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:34.712906 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:34.712909 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.712967 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.713060 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":123,"threads_started":1}
I20260809 08:46:34.713967 27248 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33115
I20260809 08:46:34.714092 27248 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:34.714325 27248 heartbeater.cc:511] Master 127.31.63.126:33115 requested a full tablet report, sending...
I20260809 08:46:34.714758 26785 ts_manager.cc:194] Registered new tserver with Master: 58143f7bce4843a69ef0360b899d5ac3 (127.31.63.66:35381)
I20260809 08:46:34.715358 26785 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.66:40641
W20260809 08:46:34.797683 27253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:34.797889 27253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:34.797911 27253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:34.799049 27253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:34.799091 27253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:34.799116 27253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:46:34.800487 27253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:40197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:33115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:34.800745 27253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:34.801057 27253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:34.806524 27259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:34.806496 27260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:34.806510 27263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:34.806890 27253 server_base.cc:1034] running on GCE node
I20260809 08:46:34.807093 27253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:34.807315 27253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:34.808481 27253 hybrid_clock.cc:648] HybridClock initialized: now 1786265194808474 us; error 41 us; skew 500 ppm
I20260809 08:46:34.809842 27253 webserver.cc:460] Webserver started at http://127.31.63.67:42285/ using document root <none> and password file <none>
I20260809 08:46:34.810099 27253 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:34.810194 27253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:34.810341 27253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:34.811444 27253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "e0f65b0e643444bb856a1d8b18601e6c"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "6b26317e172e5d2190884320b805bbfe"
server_key_iv: "02c9d29802efec1068109b37b4e9177b"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.811823 27253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "e0f65b0e643444bb856a1d8b18601e6c"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "6b26317e172e5d2190884320b805bbfe"
server_key_iv: "02c9d29802efec1068109b37b4e9177b"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.813187 27253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:34.814025 27274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.814195 27253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.814311 27253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "e0f65b0e643444bb856a1d8b18601e6c"
format_stamp: "Formatted at 2026-08-09 08:46:34 on dist-test-slave-gkw7"
server_key: "6b26317e172e5d2190884320b805bbfe"
server_key_iv: "02c9d29802efec1068109b37b4e9177b"
server_key_version: "encryptionkey@0"
I20260809 08:46:34.814388 27253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:34.823776 27253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:34.824074 27253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:34.824220 27253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:34.824446 27253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:34.824803 27253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:34.824882 27253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.824920 27253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:34.824945 27253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:34.831390 27253 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:39717
I20260809 08:46:34.831446 27436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:39717 every 8 connection(s)
I20260809 08:46:34.831847 27253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:46:34.837116 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.832584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:47125 (local address 127.31.63.126:33115)
0809 08:46:34.832672 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.832675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.832951 (+ 276us) server_negotiation.cc:407] Connection header received
0809 08:46:34.833457 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.833458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.833469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.833522 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:34.834282 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.834804 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.835808 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.836008 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.836075 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.836858 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:34.836860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:34.836860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:34.836865 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:34.836880 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:34.836885 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:34.836934 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.836995 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.837022 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:46:34.837342 27439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.832573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33115 (local address 127.31.63.67:47125)
0809 08:46:34.832822 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.832857 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.833361 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.833626 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.833630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.833643 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:34.834162 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.834174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.834893 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.834894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.835696 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.835698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.836093 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.836094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.836220 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.836725 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:46:34.836725 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:34.836753 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:46:34.836776 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:34.836936 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:34.836939 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:34.836941 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.836986 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.837060 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":110,"threads_started":1}
I20260809 08:46:34.837333 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 27253
I20260809 08:46:34.837448 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:46:34.837713 31997 external_mini_cluster.cc:1428] Setting key 410c1b543d04770bbaa2690a922f91d4
I20260809 08:46:34.837828 27437 heartbeater.cc:347] Connected to a master server at 127.31.63.126:33115
I20260809 08:46:34.837929 27437 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:34.838116 27437 heartbeater.cc:511] Master 127.31.63.126:33115 requested a full tablet report, sending...
I20260809 08:46:34.838526 26784 ts_manager.cc:194] Registered new tserver with Master: e0f65b0e643444bb856a1d8b18601e6c (127.31.63.67:39717)
I20260809 08:46:34.838860 26784 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.67:47125
I20260809 08:46:34.849793 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:46:34.854467 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.851364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46578 (local address 127.31.63.126:33115)
0809 08:46:34.851415 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.851417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.851509 (+ 92us) server_negotiation.cc:407] Connection header received
0809 08:46:34.851550 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.851551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.851561 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.851623 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:34.851781 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.852273 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.853021 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.853249 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.853329 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.854152 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:34.854153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:34.854153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:34.854158 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:34.854171 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:34.854175 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:34.854268 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.854396 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.854407 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:46:34.856364 26784 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:46:34.858109 26784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:46:34.865064 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.860953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37346 (local address 127.31.63.66:35381)
0809 08:46:34.861166 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.861172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.861183 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:46:34.861225 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.861227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.861249 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.861343 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:34.861637 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.862454 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.863768 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.864605 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.864714 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.864818 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.864918 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.864960 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
I20260809 08:46:34.865082 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.860658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:35381 (local address 127.0.0.1:37346)
0809 08:46:34.860962 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.860975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.861034 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.861433 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.861436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.861439 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:34.861522 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.861548 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.862544 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.862546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.863674 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.863677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.864717 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.864718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.864805 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.864818 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.864902 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.864960 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":124,"threads_started":1}
I20260809 08:46:34.865201 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.860674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:39717 (local address 127.0.0.1:59094)
0809 08:46:34.861086 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.861095 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.861148 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.861434 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.861437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.861440 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:34.861521 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.861523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.863042 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.863043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.864022 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.864024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.864859 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.864860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.864920 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.864924 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.864972 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.865012 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":43,"thread_start_us":113,"threads_started":1}
I20260809 08:46:34.865273 27450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.860944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59094 (local address 127.31.63.67:39717)
0809 08:46:34.861171 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.861175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.861185 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:46:34.861221 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.861222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.861256 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.861344 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:34.861682 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.862947 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.864105 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.864669 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.864739 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.864824 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.865121 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.865164 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":104,"threads_started":1}
I20260809 08:46:34.866122 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.860650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35239 (local address 127.0.0.1:57804)
0809 08:46:34.860850 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:34.860889 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:34.860967 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:34.861756 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:34.861757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:34.861759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:34.861806 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.861807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.863189 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.863190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.864427 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:34.864431 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.865913 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.865914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:34.865981 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.865984 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:46:34.866034 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:46:34.866066 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":59,"threads_started":1}
I20260809 08:46:34.866034 27148 tablet_service.cc:1467] Processing CreateTablet for tablet 1a8d2d41b57645d5b8dddb0ca758b080 (DEFAULT_TABLE table=test-table [id=95cde61407d64e2e92e9469ead41a8f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:34.866190 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:34.860750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57804 (local address 127.31.63.65:35239)
0809 08:46:34.861499 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:34.861504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:34.861513 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:46:34.861572 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:34.861574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:34.861591 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:34.861678 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:34.861899 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.863084 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.864527 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:34.865837 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:34.865914 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:34.865996 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:34.866051 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 08:46:34.866092 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":90,"threads_started":1}
I20260809 08:46:34.866429 27148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a8d2d41b57645d5b8dddb0ca758b080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:34.869096 27452 tablet_bootstrap.cc:492] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: Bootstrap starting.
I20260809 08:46:34.869469 27343 tablet_service.cc:1467] Processing CreateTablet for tablet 1a8d2d41b57645d5b8dddb0ca758b080 (DEFAULT_TABLE table=test-table [id=95cde61407d64e2e92e9469ead41a8f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:34.869905 27343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a8d2d41b57645d5b8dddb0ca758b080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:34.869939 27452 tablet_bootstrap.cc:654] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:34.869961 26960 tablet_service.cc:1467] Processing CreateTablet for tablet 1a8d2d41b57645d5b8dddb0ca758b080 (DEFAULT_TABLE table=test-table [id=95cde61407d64e2e92e9469ead41a8f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:34.870323 27452 log.cc:824] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:34.870349 26960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a8d2d41b57645d5b8dddb0ca758b080. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:34.871219 27452 tablet_bootstrap.cc:492] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: No bootstrap required, opened a new log
I20260809 08:46:34.871301 27452 ts_tablet_manager.cc:1397] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:46:34.872789 27454 tablet_bootstrap.cc:492] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: Bootstrap starting.
I20260809 08:46:34.873445 27454 tablet_bootstrap.cc:654] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:34.873792 27454 log.cc:824] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:34.874221 27459 tablet_bootstrap.cc:492] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c: Bootstrap starting.
I20260809 08:46:34.874835 27454 tablet_bootstrap.cc:492] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: No bootstrap required, opened a new log
I20260809 08:46:34.874874 27459 tablet_bootstrap.cc:654] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:34.874774 27452 raft_consensus.cc:348] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 } } peers { permanent_uuid: "acba4c67ece747f0b5acd876d153257c" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35239 } } peers { permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 } }
I20260809 08:46:34.874915 27454 ts_tablet_manager.cc:1397] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:46:34.874951 27452 raft_consensus.cc:374] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:34.875003 27452 raft_consensus.cc:729] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58143f7bce4843a69ef0360b899d5ac3, State: Initialized, Role: FOLLOWER
I20260809 08:46:34.875130 27452 consensus_queue.cc:260] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 } } peers { permanent_uuid: "acba4c67ece747f0b5acd876d153257c" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35239 } } peers { permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 } }
I20260809 08:46:34.875247 27459 log.cc:824] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:34.875422 27452 ts_tablet_manager.cc:1428] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 08:46:34.875562 27248 heartbeater.cc:503] Master 127.31.63.126:33115 was elected leader, sending a full tablet report...
I20260809 08:46:34.875947 27459 tablet_bootstrap.cc:492] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c: No bootstrap required, opened a new log
I20260809 08:46:34.876003 27459 ts_tablet_manager.cc:1397] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:46:34.877633 27454 raft_consensus.cc:348] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 } } peers { permanent_uuid: "acba4c67ece747f0b5acd876d153257c" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35239 } } peers { permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 } }
I20260809 08:46:34.877868 27454 raft_consensus.cc:374] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:34.877934 27454 raft_consensus.cc:729] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0f65b0e643444bb856a1d8b18601e6c, State: Initialized, Role: FOLLOWER
I20260809 08:46:34.878072 27454 consensus_queue.cc:260] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 } } peers { permanent_uuid: "acba4c67ece747f0b5acd876d153257c" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35239 } } peers { permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 } }
I20260809 08:46:34.878392 27454 ts_tablet_manager.cc:1428] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260809 08:46:34.878415 27437 heartbeater.cc:503] Master 127.31.63.126:33115 was elected leader, sending a full tablet report...
I20260809 08:46:34.878473 27459 raft_consensus.cc:348] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 } } peers { permanent_uuid: "acba4c67ece747f0b5acd876d153257c" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35239 } } peers { permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 } }
I20260809 08:46:34.878630 27459 raft_consensus.cc:374] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:34.878687 27459 raft_consensus.cc:729] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acba4c67ece747f0b5acd876d153257c, State: Initialized, Role: FOLLOWER
I20260809 08:46:34.878825 27459 consensus_queue.cc:260] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 } } peers { permanent_uuid: "acba4c67ece747f0b5acd876d153257c" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35239 } } peers { permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 } }
I20260809 08:46:34.879103 27459 ts_tablet_manager.cc:1428] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:46:34.879179 27057 heartbeater.cc:503] Master 127.31.63.126:33115 was elected leader, sending a full tablet report...
W20260809 08:46:34.957077 27249 tablet.cc:2369] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:46:35.079144 27058 tablet.cc:2369] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:46:35.082403 27438 tablet.cc:2369] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:46:35.188024 27468 raft_consensus.cc:482] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:35.188174 27468 raft_consensus.cc:504] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 } } peers { permanent_uuid: "acba4c67ece747f0b5acd876d153257c" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35239 } } peers { permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 } }
I20260809 08:46:35.188704 27468 leader_election.cc:288] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58143f7bce4843a69ef0360b899d5ac3 (127.31.63.66:35381), e0f65b0e643444bb856a1d8b18601e6c (127.31.63.67:39717)
I20260809 08:46:35.192382 27450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:35.188988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:51813 (local address 127.31.63.67:39717)
0809 08:46:35.189037 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:35.189039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:35.189153 (+ 114us) server_negotiation.cc:407] Connection header received
0809 08:46:35.189217 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:35.189218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:35.189229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:35.189290 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:35.189486 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:35.190317 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:35.191401 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:35.191955 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:35.192036 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:35.192046 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:35.192273 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 08:46:35.192314 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:46:35.192382 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:35.188984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:59069 (local address 127.31.63.66:35381)
0809 08:46:35.189034 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:35.189037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:35.189190 (+ 153us) server_negotiation.cc:407] Connection header received
0809 08:46:35.189263 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:35.189282 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:35.189292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:35.189352 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:35.189603 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:35.190308 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:35.191492 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:35.192026 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:35.192102 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:35.192109 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:35.192267 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 08:46:35.192307 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:46:35.192420 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:35.188925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:35381 (local address 127.31.63.65:59069)
0809 08:46:35.189071 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:35.189084 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:35.189181 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:35.189379 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:35.189380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:35.189381 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:35.189479 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:35.189481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:35.190386 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:35.190387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:35.191404 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:35.191407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:35.192109 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:35.192110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:35.192187 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:35.192190 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:46:35.192248 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:46:35.192302 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":9,"thread_start_us":66,"threads_started":1}
I20260809 08:46:35.192435 27471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:35.188917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:39717 (local address 127.31.63.65:51813)
0809 08:46:35.189059 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:35.189070 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:35.189143 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:35.189307 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:35.189308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:35.189311 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:35.189397 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:35.189400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:35.190429 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:35.190430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:35.191312 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:35.191315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:35.192045 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:35.192046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:35.192174 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:35.192181 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:46:35.192248 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:46:35.192302 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":4,"thread_start_us":81,"threads_started":1}
I20260809 08:46:35.192845 27372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a8d2d41b57645d5b8dddb0ca758b080" candidate_uuid: "acba4c67ece747f0b5acd876d153257c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0f65b0e643444bb856a1d8b18601e6c" is_pre_election: true
I20260809 08:46:35.192912 27184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a8d2d41b57645d5b8dddb0ca758b080" candidate_uuid: "acba4c67ece747f0b5acd876d153257c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58143f7bce4843a69ef0360b899d5ac3" is_pre_election: true
I20260809 08:46:35.193019 27372 raft_consensus.cc:2454] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acba4c67ece747f0b5acd876d153257c in term 0.
I20260809 08:46:35.193090 27184 raft_consensus.cc:2454] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acba4c67ece747f0b5acd876d153257c in term 0.
I20260809 08:46:35.193272 26899 leader_election.cc:302] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: acba4c67ece747f0b5acd876d153257c, e0f65b0e643444bb856a1d8b18601e6c; no voters:
I20260809 08:46:35.193434 27468 raft_consensus.cc:2781] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:46:35.193522 27468 raft_consensus.cc:482] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:46:35.193568 27468 raft_consensus.cc:3037] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:35.194362 27468 raft_consensus.cc:504] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 } } peers { permanent_uuid: "acba4c67ece747f0b5acd876d153257c" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35239 } } peers { permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 } }
I20260809 08:46:35.194525 27468 leader_election.cc:288] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [CANDIDATE]: Term 1 election: Requested vote from peers 58143f7bce4843a69ef0360b899d5ac3 (127.31.63.66:35381), e0f65b0e643444bb856a1d8b18601e6c (127.31.63.67:39717)
I20260809 08:46:35.194697 27372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a8d2d41b57645d5b8dddb0ca758b080" candidate_uuid: "acba4c67ece747f0b5acd876d153257c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0f65b0e643444bb856a1d8b18601e6c"
I20260809 08:46:35.194718 27184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a8d2d41b57645d5b8dddb0ca758b080" candidate_uuid: "acba4c67ece747f0b5acd876d153257c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58143f7bce4843a69ef0360b899d5ac3"
I20260809 08:46:35.194799 27184 raft_consensus.cc:3037] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:35.194811 27372 raft_consensus.cc:3037] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:35.195868 27184 raft_consensus.cc:2454] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acba4c67ece747f0b5acd876d153257c in term 1.
I20260809 08:46:35.195868 27372 raft_consensus.cc:2454] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acba4c67ece747f0b5acd876d153257c in term 1.
I20260809 08:46:35.196094 26900 leader_election.cc:302] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58143f7bce4843a69ef0360b899d5ac3, acba4c67ece747f0b5acd876d153257c; no voters:
I20260809 08:46:35.196192 27468 raft_consensus.cc:2781] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:35.196362 27468 raft_consensus.cc:686] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 1 LEADER]: Becoming Leader. State: Replica: acba4c67ece747f0b5acd876d153257c, State: Running, Role: LEADER
I20260809 08:46:35.196477 27468 consensus_queue.cc:237] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 } } peers { permanent_uuid: "acba4c67ece747f0b5acd876d153257c" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35239 } } peers { permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 } }
I20260809 08:46:35.197387 26784 catalog_manager.cc:5495] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c reported cstate change: term changed from 0 to 1, leader changed from <none> to acba4c67ece747f0b5acd876d153257c (127.31.63.65). New cstate: current_term: 1 leader_uuid: "acba4c67ece747f0b5acd876d153257c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acba4c67ece747f0b5acd876d153257c" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 35239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 } health_report { overall_health: UNKNOWN } } }
I20260809 08:46:35.218631 27449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:35.216033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57806 (local address 127.31.63.65:35239)
0809 08:46:35.216106 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:35.216109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:35.216119 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:46:35.216184 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:35.216185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:35.216207 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:35.216286 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:46:35.216473 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:35.216962 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:35.217863 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:35.218071 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:35.218141 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:35.218275 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:46:35.218416 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:46:35.218504 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:35.218555 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 08:46:35.218567 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 08:46:35.220773 27372 raft_consensus.cc:1261] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c [term 1 FOLLOWER]: Refusing update from remote peer acba4c67ece747f0b5acd876d153257c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:35.220773 27184 raft_consensus.cc:1261] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3 [term 1 FOLLOWER]: Refusing update from remote peer acba4c67ece747f0b5acd876d153257c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:35.221212 27468 consensus_queue.cc:1035] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0f65b0e643444bb856a1d8b18601e6c" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 39717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:35.221376 27472 consensus_queue.cc:1035] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [LEADER]: Connected to new peer: Peer: permanent_uuid: "58143f7bce4843a69ef0360b899d5ac3" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:35.223838 27476 mvcc.cc:205] Tried to move back new op lower bound from 7316542239620849664 to 7316542239525367808. Current Snapshot: MvccSnapshot[applied={T|T < 7316542239620849664}]
I20260809 08:46:35.224109 27477 mvcc.cc:205] Tried to move back new op lower bound from 7316542239620849664 to 7316542239525367808. Current Snapshot: MvccSnapshot[applied={T|T < 7316542239620849664}]
I20260809 08:46:35.226504 26785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:35.226621 26785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:35.227442 26785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6460e150a0d49329ebdf2ce51edf1d0: Sending DeleteTablet for 3 replicas of tablet 1a8d2d41b57645d5b8dddb0ca758b080
I20260809 08:46:35.227773 27148 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a8d2d41b57645d5b8dddb0ca758b080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:35 UTC) from {username='slave'} at 127.0.0.1:37346
I20260809 08:46:35.227852 27343 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a8d2d41b57645d5b8dddb0ca758b080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:35 UTC) from {username='slave'} at 127.0.0.1:59094
I20260809 08:46:35.227854 26960 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a8d2d41b57645d5b8dddb0ca758b080 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:35 UTC) from {username='slave'} at 127.0.0.1:57804
I20260809 08:46:35.228070 27489 tablet_replica.cc:330] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: stopping tablet replica
I20260809 08:46:35.228214 27489 raft_consensus.cc:2229] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:35.228215 27491 tablet_replica.cc:330] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c: stopping tablet replica
I20260809 08:46:35.228334 27489 raft_consensus.cc:2258] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:35.228341 27491 raft_consensus.cc:2229] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:46:35.228551 27491 raft_consensus.cc:2258] T 1a8d2d41b57645d5b8dddb0ca758b080 P acba4c67ece747f0b5acd876d153257c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:35.228607 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 26879
I20260809 08:46:35.228857 27489 ts_tablet_manager.cc:1905] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:35.229348 27488 tablet_replica.cc:330] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: stopping tablet replica
I20260809 08:46:35.229525 27488 raft_consensus.cc:2229] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:35.229718 27488 raft_consensus.cc:2258] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:35.231823 27488 ts_tablet_manager.cc:1905] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:35.231845 27489 ts_tablet_manager.cc:1918] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:35.231918 27489 log.cc:1196] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/1a8d2d41b57645d5b8dddb0ca758b080
I20260809 08:46:35.232198 27489 ts_tablet_manager.cc:1939] T 1a8d2d41b57645d5b8dddb0ca758b080 P e0f65b0e643444bb856a1d8b18601e6c: Deleting consensus metadata
I20260809 08:46:35.232609 26763 catalog_manager.cc:4841] TS e0f65b0e643444bb856a1d8b18601e6c (127.31.63.67:39717): tablet 1a8d2d41b57645d5b8dddb0ca758b080 (table test-table [id=95cde61407d64e2e92e9469ead41a8f5]) successfully deleted
I20260809 08:46:35.233240 27488 ts_tablet_manager.cc:1918] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:35.233331 27488 log.cc:1196] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/1a8d2d41b57645d5b8dddb0ca758b080
I20260809 08:46:35.233624 27488 ts_tablet_manager.cc:1939] T 1a8d2d41b57645d5b8dddb0ca758b080 P 58143f7bce4843a69ef0360b899d5ac3: Deleting consensus metadata
I20260809 08:46:35.234032 26764 catalog_manager.cc:4841] TS 58143f7bce4843a69ef0360b899d5ac3 (127.31.63.66:35381): tablet 1a8d2d41b57645d5b8dddb0ca758b080 (table test-table [id=95cde61407d64e2e92e9469ead41a8f5]) successfully deleted
W20260809 08:46:35.241919 26765 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:35239 (error 108)
I20260809 08:46:35.242141 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 27061
I20260809 08:46:35.246538 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:35.242282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:35239 (local address 127.0.0.1:57818)
0809 08:46:35.246425 (+ 4143us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:35.246489 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:35239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4115}
W20260809 08:46:35.248514 26765 catalog_manager.cc:4584] TS acba4c67ece747f0b5acd876d153257c (127.31.63.65:35239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1a8d2d41b57645d5b8dddb0ca758b080: Network error: Client connection negotiation failed: client connection to 127.31.63.65:35239: connect: Connection refused (error 111)
I20260809 08:46:35.253895 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 27253
I20260809 08:46:35.266129 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 26740
2026-08-09T08:46:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1026 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:35 dist-test-slave-gkw7 krb5kdc[27499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:35 dist-test-slave-gkw7 krb5kdc[27499](info): set up 2 sockets
Aug 09 08:46:35 dist-test-slave-gkw7 krb5kdc[27499](info): commencing operation
krb5kdc: starting...
W20260809 08:46:36.531111 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:36 dist-test-slave-gkw7 krb5kdc[27499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:46:36.658676 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:44149
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:46305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:44149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:36.741045 27520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:36.741293 27520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:36.741331 27520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:36.742498 27520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:36.742564 27520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:36.742587 27520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:36.742619 27520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:36.742664 27520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:36.744145 27520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:44149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:44149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:36.744421 27520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:36.744654 27520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:36.749686 27527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:36.749660 27526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:36.749688 27530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:36.749864 27520 server_base.cc:1034] running on GCE node
I20260809 08:46:36.750144 27520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:36.750444 27520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:36.751663 27520 hybrid_clock.cc:648] HybridClock initialized: now 1786265196751640 us; error 38 us; skew 500 ppm
Aug 09 08:46:36 dist-test-slave-gkw7 krb5kdc[27499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265196, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:36.754459 27520 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:46:36.754925 27520 webserver.cc:460] Webserver started at http://127.31.63.126:35609/ using document root <none> and password file <none>
I20260809 08:46:36.755172 27520 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:36.755259 27520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:36.755422 27520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:36.756569 27520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "e8497cc1e6ce412dbb610732b9d4b466"
format_stamp: "Formatted at 2026-08-09 08:46:36 on dist-test-slave-gkw7"
server_key: "a2ee2f87c5293122a63750efc33eecc5"
server_key_iv: "e37ed0b30709cc82cfe0fbdc0b935e68"
server_key_version: "encryptionkey@0"
I20260809 08:46:36.756970 27520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "e8497cc1e6ce412dbb610732b9d4b466"
format_stamp: "Formatted at 2026-08-09 08:46:36 on dist-test-slave-gkw7"
server_key: "a2ee2f87c5293122a63750efc33eecc5"
server_key_iv: "e37ed0b30709cc82cfe0fbdc0b935e68"
server_key_version: "encryptionkey@0"
I20260809 08:46:36.758576 27520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:46:36.759425 27537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:36.759626 27520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:36.759740 27520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "e8497cc1e6ce412dbb610732b9d4b466"
format_stamp: "Formatted at 2026-08-09 08:46:36 on dist-test-slave-gkw7"
server_key: "a2ee2f87c5293122a63750efc33eecc5"
server_key_iv: "e37ed0b30709cc82cfe0fbdc0b935e68"
server_key_version: "encryptionkey@0"
I20260809 08:46:36.759828 27520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:36.786674 27520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:36.788983 27520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:36.789177 27520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:36.793879 27520 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:44149
I20260809 08:46:36.793922 27603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:44149 every 8 connection(s)
I20260809 08:46:36.794340 27520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:36.795348 27604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:36.797837 27604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466: Bootstrap starting.
I20260809 08:46:36.798477 27604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:36.798781 27604 log.cc:824] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:36.799558 27604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466: No bootstrap required, opened a new log
I20260809 08:46:36.801257 27604 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8497cc1e6ce412dbb610732b9d4b466" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44149 } }
I20260809 08:46:36.801409 27604 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:36.801435 27604 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8497cc1e6ce412dbb610732b9d4b466, State: Initialized, Role: FOLLOWER
I20260809 08:46:36.801617 27604 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8497cc1e6ce412dbb610732b9d4b466" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44149 } }
I20260809 08:46:36.801710 27604 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:36.801735 27604 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:36.801803 27604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:36.802380 27604 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8497cc1e6ce412dbb610732b9d4b466" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44149 } }
I20260809 08:46:36.802470 27604 leader_election.cc:302] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8497cc1e6ce412dbb610732b9d4b466; no voters:
I20260809 08:46:36.802666 27604 leader_election.cc:288] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:36.802784 27609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:36.802974 27609 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [term 1 LEADER]: Becoming Leader. State: Replica: e8497cc1e6ce412dbb610732b9d4b466, State: Running, Role: LEADER
I20260809 08:46:36.803020 27604 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:36.803081 27609 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8497cc1e6ce412dbb610732b9d4b466" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44149 } }
I20260809 08:46:36.803637 27610 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8497cc1e6ce412dbb610732b9d4b466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8497cc1e6ce412dbb610732b9d4b466" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44149 } } }
I20260809 08:46:36.803788 27612 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8497cc1e6ce412dbb610732b9d4b466. Latest consensus state: current_term: 1 leader_uuid: "e8497cc1e6ce412dbb610732b9d4b466" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8497cc1e6ce412dbb610732b9d4b466" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44149 } } }
I20260809 08:46:36.803853 27610 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:36.803870 27612 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:36.804145 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 27520
I20260809 08:46:36.804289 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:36.804522 31997 external_mini_cluster.cc:1428] Setting key 88c405adef031b088c1d7ac5e914c6ef
W20260809 08:46:36.805013 27631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:46:36.805070 27631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:46:36.805148 27626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:36.805332 27626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:36.806975 27626 catalog_manager.cc:1347] Generated new cluster ID: ef697f76628b427385e952f6d892f21f
I20260809 08:46:36.807029 27626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:46:36 dist-test-slave-gkw7 krb5kdc[27499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:36.815179 27634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:36.805049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.31.63.126:44149)
0809 08:46:36.805262 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:36.805272 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:36.805758 (+ 486us) server_negotiation.cc:407] Connection header received
0809 08:46:36.805881 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:36.805888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:36.805951 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:36.806106 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:36.806951 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:36.807944 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:36.808747 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:36.809193 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:36.809326 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:36.811759 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:36.811763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:36.811763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:36.811773 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:36.813496 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:36.813924 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:36.813928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:36.813928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:36.814050 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:36.814292 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:36.814293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:36.814293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:36.814536 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:36.814711 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:36.814830 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 08:46:36.814978 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":127,"threads_started":1}
I20260809 08:46:36.827734 27626 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:36.828325 27626 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:36.832882 27626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466: Generated new TSK 0
I20260809 08:46:36.833092 27626 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
Entry for principal kudu/127.31.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab.
I20260809 08:46:36.868978 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:44149
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:46305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:36.967504 27639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:36.967729 27639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:36.967756 27639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:36.968938 27639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:36.968982 27639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:36.969009 27639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:46:36.970486 27639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.65.keytab
--principal=kudu/127.31.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:44149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:36.970742 27639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:36.970983 27639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:36.976315 27653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:36.976502 27648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:36.976508 27650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:36.977025 27639 server_base.cc:1034] running on GCE node
I20260809 08:46:36.977315 27639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:36.977665 27639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:36.978864 27639 hybrid_clock.cc:648] HybridClock initialized: now 1786265196978830 us; error 51 us; skew 500 ppm
Aug 09 08:46:36 dist-test-slave-gkw7 krb5kdc[27499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265196, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:36.981777 27639 init.cc:377] Logged in from keytab as kudu/127.31.63.65@KRBTEST.COM (short username kudu)
I20260809 08:46:36.982285 27639 webserver.cc:460] Webserver started at http://127.31.63.65:46071/ using document root <none> and password file <none>
I20260809 08:46:36.982542 27639 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:36.982601 27639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:36.982800 27639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:36.983958 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "e5b65a252de54eb0b33d6fd056f27c71"
format_stamp: "Formatted at 2026-08-09 08:46:36 on dist-test-slave-gkw7"
server_key: "1382650e8a88b195d247007b3b2b2263"
server_key_iv: "8c5e13fcf11de3e5307723a661b3cf9a"
server_key_version: "encryptionkey@0"
I20260809 08:46:36.984354 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "e5b65a252de54eb0b33d6fd056f27c71"
format_stamp: "Formatted at 2026-08-09 08:46:36 on dist-test-slave-gkw7"
server_key: "1382650e8a88b195d247007b3b2b2263"
server_key_iv: "8c5e13fcf11de3e5307723a661b3cf9a"
server_key_version: "encryptionkey@0"
I20260809 08:46:36.986006 27639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:36.986904 27665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:36.987131 27639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:36.987233 27639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "e5b65a252de54eb0b33d6fd056f27c71"
format_stamp: "Formatted at 2026-08-09 08:46:36 on dist-test-slave-gkw7"
server_key: "1382650e8a88b195d247007b3b2b2263"
server_key_iv: "8c5e13fcf11de3e5307723a661b3cf9a"
server_key_version: "encryptionkey@0"
I20260809 08:46:36.987317 27639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:37.000938 27639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:37.003547 27639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:37.003769 27639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:37.004104 27639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:37.004438 27639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:37.004474 27639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.004501 27639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:37.004516 27639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.011349 27639 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:41655
I20260809 08:46:37.011439 27828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:41655 every 8 connection(s)
I20260809 08:46:37.011834 27639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:46:37.016072 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 27639
I20260809 08:46:37.016230 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:46:37.016453 31997 external_mini_cluster.cc:1428] Setting key 39a84f24a0a29bbff86d2a5111010849
Aug 09 08:46:37 dist-test-slave-gkw7 krb5kdc[27499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265196, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.65@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:37.023348 27634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.012715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:47105 (local address 127.31.63.126:44149)
0809 08:46:37.012809 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.012812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.013011 (+ 199us) server_negotiation.cc:407] Connection header received
0809 08:46:37.013669 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.013671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.013688 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.013749 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:37.015274 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.015866 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.017334 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.017708 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.017785 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.020276 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:37.020278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:37.020278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:37.020286 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:37.021841 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:37.022320 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:37.022321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:37.022322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:37.022384 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:37.022652 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:37.022653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:37.022653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:37.022773 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:37.022842 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.023138 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.023253 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 08:46:37.023550 27832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.012660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:44149 (local address 127.31.63.65:47105)
0809 08:46:37.012870 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:37.012912 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:37.013492 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:37.013829 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:37.013833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:37.014852 (+ 1019us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:37.015150 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.015157 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.015983 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.015984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.017204 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.017207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.017801 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.017803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.017920 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.018533 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:37.018542 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:37.020169 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:37.021939 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:37.021948 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:37.021950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:37.022238 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:37.022460 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:37.022461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:37.022461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:37.022576 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:37.022846 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:37.022855 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:37.022999 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 08:46:37.023114 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 08:46:37.023284 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
I20260809 08:46:37.024019 27830 heartbeater.cc:347] Connected to a master server at 127.31.63.126:44149
I20260809 08:46:37.024121 27830 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:37.024331 27830 heartbeater.cc:511] Master 127.31.63.126:44149 requested a full tablet report, sending...
I20260809 08:46:37.024894 27557 ts_manager.cc:194] Registered new tserver with Master: e5b65a252de54eb0b33d6fd056f27c71 (127.31.63.65:41655)
I20260809 08:46:37.025571 27557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.65@KRBTEST.COM'} at 127.31.63.65:47105
WARNING: no policy specified for kudu/127.31.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
Entry for principal kudu/127.31.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab.
I20260809 08:46:37.045336 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:44149
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:46305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:37.130254 27835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:37.130573 27835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:37.130628 27835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:37.131832 27835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:37.131912 27835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:37.131956 27835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:46:37.133474 27835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.66.keytab
--principal=kudu/127.31.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:44149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:37.133795 27835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:37.134042 27835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:37.139521 27845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:37.139505 27844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:37.139506 27856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:37.139904 27835 server_base.cc:1034] running on GCE node
I20260809 08:46:37.140127 27835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:37.140394 27835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:37.141649 27835 hybrid_clock.cc:648] HybridClock initialized: now 1786265197141625 us; error 31 us; skew 500 ppm
Aug 09 08:46:37 dist-test-slave-gkw7 krb5kdc[27499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265197, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:37.144477 27835 init.cc:377] Logged in from keytab as kudu/127.31.63.66@KRBTEST.COM (short username kudu)
I20260809 08:46:37.144879 27835 webserver.cc:460] Webserver started at http://127.31.63.66:39657/ using document root <none> and password file <none>
I20260809 08:46:37.145061 27835 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:37.145113 27835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:37.145205 27835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:37.146281 27835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "50a6ad01baa64166a0eeefb1cfd3f527"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "bc147fca67427e92cac172ac2f685a15"
server_key_iv: "6f9d0ac8778185a8ee21fdcce0e8f9ba"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.146595 27835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "50a6ad01baa64166a0eeefb1cfd3f527"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "bc147fca67427e92cac172ac2f685a15"
server_key_iv: "6f9d0ac8778185a8ee21fdcce0e8f9ba"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.147905 27835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:46:37.148677 27864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.148862 27835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:37.148954 27835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "50a6ad01baa64166a0eeefb1cfd3f527"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "bc147fca67427e92cac172ac2f685a15"
server_key_iv: "6f9d0ac8778185a8ee21fdcce0e8f9ba"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.149010 27835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:37.161402 27835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:37.163648 27835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:37.163769 27835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:37.163973 27835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:37.164314 27835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:37.164348 27835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.164371 27835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:37.164417 27835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.170480 27835 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:40795
I20260809 08:46:37.170564 28022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:40795 every 8 connection(s)
I20260809 08:46:37.170907 27835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:46:37.171175 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 27835
I20260809 08:46:37.171321 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:46:37.171581 31997 external_mini_cluster.cc:1428] Setting key 963e55e04d6854b8e0eb58860542703f
Aug 09 08:46:37 dist-test-slave-gkw7 krb5kdc[27499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265197, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.66@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.63.67@KRBTEST.COM; defaulting to no policy
I20260809 08:46:37.182193 27634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.171703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:46965 (local address 127.31.63.126:44149)
0809 08:46:37.171755 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.171758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.172074 (+ 316us) server_negotiation.cc:407] Connection header received
0809 08:46:37.172801 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.172802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.172815 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.172877 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:37.173971 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.174785 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.175858 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.176201 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.176278 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.179007 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:37.179009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:37.179010 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:37.179019 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:37.180683 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:37.181084 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:37.181086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:37.181087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:37.181164 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:37.181423 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:37.181425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:37.181425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:37.181608 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:37.181687 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.181952 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.182050 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:46:37.182328 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.171667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:44149 (local address 127.31.63.66:46965)
0809 08:46:37.171942 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:37.171999 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:37.172686 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:37.172953 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:37.172957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:37.173649 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:37.173865 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.173870 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.174906 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.174907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.175771 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.175773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.176297 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.176298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.176418 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.176998 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:37.177007 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:37.178905 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:37.180771 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:37.180778 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:37.180780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:37.181005 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:37.181242 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:37.181243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:37.181243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:37.181348 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:37.181678 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:37.181684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:37.181818 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 08:46:37.181906 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 08:46:37.182035 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":82,"threads_started":1}
I20260809 08:46:37.182895 28023 heartbeater.cc:347] Connected to a master server at 127.31.63.126:44149
I20260809 08:46:37.183012 28023 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:37.183234 28023 heartbeater.cc:511] Master 127.31.63.126:44149 requested a full tablet report, sending...
I20260809 08:46:37.183656 27557 ts_manager.cc:194] Registered new tserver with Master: 50a6ad01baa64166a0eeefb1cfd3f527 (127.31.63.66:40795)
I20260809 08:46:37.184043 27557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.66@KRBTEST.COM'} at 127.31.63.66:46965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
Entry for principal kudu/127.31.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab.
I20260809 08:46:37.200493 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:44149
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:46305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:37.290864 28029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:37.291071 28029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:37.291100 28029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:37.292232 28029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:37.292268 28029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:37.292294 28029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:46:37.293772 28029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.67.keytab
--principal=kudu/127.31.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:44149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:37.293970 28029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:37.294154 28029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:37.299715 28042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:37.299715 28040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:37.299755 28044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:37.299908 28029 server_base.cc:1034] running on GCE node
I20260809 08:46:37.300333 28029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:37.300596 28029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:37.301733 28029 hybrid_clock.cc:648] HybridClock initialized: now 1786265197301721 us; error 29 us; skew 500 ppm
Aug 09 08:46:37 dist-test-slave-gkw7 krb5kdc[27499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265197, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:37.304774 28029 init.cc:377] Logged in from keytab as kudu/127.31.63.67@KRBTEST.COM (short username kudu)
I20260809 08:46:37.305169 28029 webserver.cc:460] Webserver started at http://127.31.63.67:38129/ using document root <none> and password file <none>
I20260809 08:46:37.305358 28029 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:37.305399 28029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:37.305491 28029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:37.306586 28029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "015eb853dfcf460fb3c078c630d5f289"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "76fc8e4fb9f1e1ac7aef0b134b7a595e"
server_key_iv: "ef9589c5228beb9e86bacbec8164b28d"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.306895 28029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "015eb853dfcf460fb3c078c630d5f289"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "76fc8e4fb9f1e1ac7aef0b134b7a595e"
server_key_iv: "ef9589c5228beb9e86bacbec8164b28d"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.308169 28029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:46:37.308919 28051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.309173 28029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:37.309294 28029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "015eb853dfcf460fb3c078c630d5f289"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "76fc8e4fb9f1e1ac7aef0b134b7a595e"
server_key_iv: "ef9589c5228beb9e86bacbec8164b28d"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.309374 28029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:37.347888 28029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:37.350497 28029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:37.350675 28029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:37.350931 28029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:37.351276 28029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:37.351313 28029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.351336 28029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:37.351384 28029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.358022 28029 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:40947
I20260809 08:46:37.358084 28226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:40947 every 8 connection(s)
I20260809 08:46:37.358425 28029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:46:37 dist-test-slave-gkw7 krb5kdc[27499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265197, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.67@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:37.367255 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 28029
I20260809 08:46:37.367340 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:46:37.367529 31997 external_mini_cluster.cc:1428] Setting key 5cd6a46593dbcb8650c5213961507374
I20260809 08:46:37.368075 27634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.359204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:41875 (local address 127.31.63.126:44149)
0809 08:46:37.359297 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.359300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.359501 (+ 201us) server_negotiation.cc:407] Connection header received
0809 08:46:37.359981 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.359981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.359991 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.360045 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:37.360990 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.361491 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.362682 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.362888 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.362942 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.365249 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:37.365251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:37.365251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:37.365258 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:37.366538 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:37.366949 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:37.366950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:37.366950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:37.367011 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:37.367274 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:37.367275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:37.367275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:37.367444 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:37.367522 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.367865 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.367952 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 08:46:37.368322 28230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.359155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:44149 (local address 127.31.63.67:41875)
0809 08:46:37.359363 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:37.359403 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:37.359894 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:37.360108 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:37.360111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:37.360694 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:37.360909 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.360913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.361627 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.361629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.362553 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.362556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.362960 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.362960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.363101 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.363707 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:46:37.363718 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:37.365146 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:37.366628 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:37.366636 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:37.366639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:37.366869 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:37.367081 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:46:37.367082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:46:37.367083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:46:37.367196 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:46:37.367513 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:37.367520 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:37.367708 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 08:46:37.367820 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 08:46:37.368048 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":117,"threads_started":1}
I20260809 08:46:37.368845 28227 heartbeater.cc:347] Connected to a master server at 127.31.63.126:44149
I20260809 08:46:37.368958 28227 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:37.369185 28227 heartbeater.cc:511] Master 127.31.63.126:44149 requested a full tablet report, sending...
I20260809 08:46:37.369611 27556 ts_manager.cc:194] Registered new tserver with Master: 015eb853dfcf460fb3c078c630d5f289 (127.31.63.67:40947)
I20260809 08:46:37.370317 27556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.63.67@KRBTEST.COM'} at 127.31.63.67:41875
I20260809 08:46:37.379751 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:46:37.388414 27634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.381465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.31.63.126:44149)
0809 08:46:37.381569 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.381571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.381744 (+ 173us) server_negotiation.cc:407] Connection header received
0809 08:46:37.381803 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.381804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.381815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.381882 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:37.382523 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.383127 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.383880 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.384112 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.384174 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.385795 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:37.385796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:37.385797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:37.385804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:37.387282 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:37.387605 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:37.387606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:37.387606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:37.387669 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:37.387887 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:37.387887 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:37.387888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:37.388035 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:37.388158 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.388276 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.388343 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 08:46:37.390407 27556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:46:37.392321 27556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:46:37.398958 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.394829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:41655 (local address 127.0.0.1:35702)
0809 08:46:37.394939 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:37.394984 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:37.395044 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:37.395646 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:37.395649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:37.395651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:37.395726 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.395727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.396842 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.396843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.397891 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.397894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.398618 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.398619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.398705 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.398719 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:46:37.398821 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 08:46:37.398882 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":67,"threads_started":1}
I20260809 08:46:37.398967 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.395216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.31.63.65:41655)
0809 08:46:37.395447 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.395453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.395463 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:46:37.395507 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.395510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.395528 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.395806 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:37.395894 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.396717 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.398004 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.398517 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.398612 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.398678 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.398841 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.398882 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":101,"threads_started":1}
I20260809 08:46:37.399703 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.394871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40947 (local address 127.0.0.1:43684)
0809 08:46:37.395411 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:37.395422 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:37.395481 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:37.395865 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:37.395866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:37.395868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:37.395926 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.395928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.397490 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.397492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.398409 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.398411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.399436 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.399437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.399508 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.399513 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:46:37.399567 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 08:46:37.399606 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":28,"thread_start_us":100,"threads_started":1}
I20260809 08:46:37.400121 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.394980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.31.63.67:40947)
0809 08:46:37.395410 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.395414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.395477 (+ 63us) server_negotiation.cc:407] Connection header received
0809 08:46:37.395610 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.395613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.395666 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.395850 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:37.396044 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.397350 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.398512 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.399331 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.399894 (+ 563us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.399970 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.399998 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.400040 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":95,"threads_started":1}
I20260809 08:46:37.400030 27736 tablet_service.cc:1467] Processing CreateTablet for tablet fdcaacb43daa447fbf105ea044ceb8c7 (DEFAULT_TABLE table=test-table [id=cae3b763656744c3b3b3c38cdfaf4c66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:37.400447 27736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdcaacb43daa447fbf105ea044ceb8c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:37.400978 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.395050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:40795 (local address 127.0.0.1:55018)
0809 08:46:37.395564 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:37.395573 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:37.395631 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:37.396043 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:37.396044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:37.396046 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:37.396084 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.396085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.397572 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.397573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.399637 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.399639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.400720 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.400721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.400790 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.400794 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:37.400850 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 08:46:37.400894 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":26,"thread_start_us":81,"threads_started":1}
I20260809 08:46:37.400908 28132 tablet_service.cc:1467] Processing CreateTablet for tablet fdcaacb43daa447fbf105ea044ceb8c7 (DEFAULT_TABLE table=test-table [id=cae3b763656744c3b3b3c38cdfaf4c66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:37.401237 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.395160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.31.63.66:40795)
0809 08:46:37.395371 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.395377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.395694 (+ 317us) server_negotiation.cc:407] Connection header received
0809 08:46:37.395745 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.395826 (+ 81us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.395848 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.395921 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:37.396177 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.397432 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.399730 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.400528 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.400608 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.400664 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.401094 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.401141 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":122,"threads_started":1}
I20260809 08:46:37.401252 28132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdcaacb43daa447fbf105ea044ceb8c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:37.401949 27929 tablet_service.cc:1467] Processing CreateTablet for tablet fdcaacb43daa447fbf105ea044ceb8c7 (DEFAULT_TABLE table=test-table [id=cae3b763656744c3b3b3c38cdfaf4c66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:37.402304 27929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdcaacb43daa447fbf105ea044ceb8c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:37.403506 28251 tablet_bootstrap.cc:492] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71: Bootstrap starting.
I20260809 08:46:37.404345 28251 tablet_bootstrap.cc:654] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:37.405011 28251 log.cc:824] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:37.405227 28252 tablet_bootstrap.cc:492] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: Bootstrap starting.
I20260809 08:46:37.405838 28251 tablet_bootstrap.cc:492] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71: No bootstrap required, opened a new log
I20260809 08:46:37.405925 28251 ts_tablet_manager.cc:1397] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:46:37.406020 28252 tablet_bootstrap.cc:654] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:37.406387 28254 tablet_bootstrap.cc:492] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: Bootstrap starting.
I20260809 08:46:37.406695 28252 log.cc:824] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:37.407109 28254 tablet_bootstrap.cc:654] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:37.407648 28252 tablet_bootstrap.cc:492] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: No bootstrap required, opened a new log
I20260809 08:46:37.407738 28254 log.cc:824] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:37.407789 28252 ts_tablet_manager.cc:1397] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 08:46:37.407927 28251 raft_consensus.cc:348] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b65a252de54eb0b33d6fd056f27c71" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41655 } } peers { permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 } } peers { permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 } }
I20260809 08:46:37.408141 28251 raft_consensus.cc:374] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:37.408188 28251 raft_consensus.cc:729] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b65a252de54eb0b33d6fd056f27c71, State: Initialized, Role: FOLLOWER
I20260809 08:46:37.408346 28251 consensus_queue.cc:260] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b65a252de54eb0b33d6fd056f27c71" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41655 } } peers { permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 } } peers { permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 } }
I20260809 08:46:37.408629 28254 tablet_bootstrap.cc:492] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: No bootstrap required, opened a new log
I20260809 08:46:37.408654 28251 ts_tablet_manager.cc:1428] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:46:37.408707 28254 ts_tablet_manager.cc:1397] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:46:37.408792 27830 heartbeater.cc:503] Master 127.31.63.126:44149 was elected leader, sending a full tablet report...
I20260809 08:46:37.410197 28252 raft_consensus.cc:348] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b65a252de54eb0b33d6fd056f27c71" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41655 } } peers { permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 } } peers { permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 } }
I20260809 08:46:37.410391 28252 raft_consensus.cc:374] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:37.410432 28252 raft_consensus.cc:729] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015eb853dfcf460fb3c078c630d5f289, State: Initialized, Role: FOLLOWER
I20260809 08:46:37.410597 28252 consensus_queue.cc:260] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b65a252de54eb0b33d6fd056f27c71" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41655 } } peers { permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 } } peers { permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 } }
I20260809 08:46:37.410915 28252 ts_tablet_manager.cc:1428] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:46:37.410930 28227 heartbeater.cc:503] Master 127.31.63.126:44149 was elected leader, sending a full tablet report...
I20260809 08:46:37.410979 28254 raft_consensus.cc:348] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b65a252de54eb0b33d6fd056f27c71" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41655 } } peers { permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 } } peers { permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 } }
I20260809 08:46:37.411168 28254 raft_consensus.cc:374] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:37.411196 28254 raft_consensus.cc:729] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50a6ad01baa64166a0eeefb1cfd3f527, State: Initialized, Role: FOLLOWER
I20260809 08:46:37.411394 28254 consensus_queue.cc:260] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b65a252de54eb0b33d6fd056f27c71" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41655 } } peers { permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 } } peers { permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 } }
I20260809 08:46:37.411759 28254 ts_tablet_manager.cc:1428] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:46:37.411785 28023 heartbeater.cc:503] Master 127.31.63.126:44149 was elected leader, sending a full tablet report...
W20260809 08:46:37.421701 28024 tablet.cc:2369] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:46:37.512732 27831 tablet.cc:2369] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:46:37.588965 28257 raft_consensus.cc:482] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:37.589115 28257 raft_consensus.cc:504] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b65a252de54eb0b33d6fd056f27c71" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41655 } } peers { permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 } } peers { permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 } }
I20260809 08:46:37.589447 28257 leader_election.cc:288] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015eb853dfcf460fb3c078c630d5f289 (127.31.63.67:40947), 50a6ad01baa64166a0eeefb1cfd3f527 (127.31.63.66:40795)
I20260809 08:46:37.593801 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.589820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:53077 (local address 127.31.63.66:40795)
0809 08:46:37.589890 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.589892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.590019 (+ 127us) server_negotiation.cc:407] Connection header received
0809 08:46:37.590077 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.590078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.590088 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.590154 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:37.590385 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.591174 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.592754 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.593319 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.593403 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.593416 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.593644 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.593689 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:46:37.593789 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.589817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:52857 (local address 127.31.63.67:40947)
0809 08:46:37.589873 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.589875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.590023 (+ 148us) server_negotiation.cc:407] Connection header received
0809 08:46:37.590070 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.590072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.590084 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.590154 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:37.590416 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.591200 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.592754 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.593353 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.593434 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.593443 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.593648 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.593689 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:46:37.593822 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.589719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:40947 (local address 127.31.63.65:52857)
0809 08:46:37.589911 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:37.589924 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:37.590013 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:37.590169 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:37.590170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:37.590173 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:37.590306 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.590309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.591281 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.591281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.592660 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.592664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.593430 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.593431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.593518 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.593526 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 08:46:37.593618 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:46:37.593659 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":88,"thread_start_us":71,"threads_started":1}
I20260809 08:46:37.593822 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.589719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:40795 (local address 127.31.63.65:53077)
0809 08:46:37.589919 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:37.589929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:37.590013 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:37.590169 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:37.590170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:37.590174 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:37.590311 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.590312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.591278 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.591280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.592660 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.592664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.593431 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.593432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.593518 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.593525 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:46:37.593601 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 08:46:37.593659 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":140,"thread_start_us":61,"threads_started":1}
I20260809 08:46:37.594439 28164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdcaacb43daa447fbf105ea044ceb8c7" candidate_uuid: "e5b65a252de54eb0b33d6fd056f27c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015eb853dfcf460fb3c078c630d5f289" is_pre_election: true
I20260809 08:46:37.594578 28164 raft_consensus.cc:2454] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5b65a252de54eb0b33d6fd056f27c71 in term 0.
I20260809 08:46:37.594794 27955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdcaacb43daa447fbf105ea044ceb8c7" candidate_uuid: "e5b65a252de54eb0b33d6fd056f27c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" is_pre_election: true
I20260809 08:46:37.594828 27670 leader_election.cc:302] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015eb853dfcf460fb3c078c630d5f289, e5b65a252de54eb0b33d6fd056f27c71; no voters:
I20260809 08:46:37.594916 27955 raft_consensus.cc:2454] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5b65a252de54eb0b33d6fd056f27c71 in term 0.
I20260809 08:46:37.594972 28257 raft_consensus.cc:2781] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:46:37.595029 28257 raft_consensus.cc:482] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:46:37.595046 28257 raft_consensus.cc:3037] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:37.595901 28257 raft_consensus.cc:504] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b65a252de54eb0b33d6fd056f27c71" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41655 } } peers { permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 } } peers { permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 } }
I20260809 08:46:37.596045 28257 leader_election.cc:288] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [CANDIDATE]: Term 1 election: Requested vote from peers 015eb853dfcf460fb3c078c630d5f289 (127.31.63.67:40947), 50a6ad01baa64166a0eeefb1cfd3f527 (127.31.63.66:40795)
I20260809 08:46:37.596256 28164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdcaacb43daa447fbf105ea044ceb8c7" candidate_uuid: "e5b65a252de54eb0b33d6fd056f27c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015eb853dfcf460fb3c078c630d5f289"
I20260809 08:46:37.596313 27955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdcaacb43daa447fbf105ea044ceb8c7" candidate_uuid: "e5b65a252de54eb0b33d6fd056f27c71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50a6ad01baa64166a0eeefb1cfd3f527"
I20260809 08:46:37.596344 28164 raft_consensus.cc:3037] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:37.596414 27955 raft_consensus.cc:3037] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:37.597122 28164 raft_consensus.cc:2454] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b65a252de54eb0b33d6fd056f27c71 in term 1.
I20260809 08:46:37.597266 27955 raft_consensus.cc:2454] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5b65a252de54eb0b33d6fd056f27c71 in term 1.
I20260809 08:46:37.597368 27670 leader_election.cc:302] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015eb853dfcf460fb3c078c630d5f289, e5b65a252de54eb0b33d6fd056f27c71; no voters:
I20260809 08:46:37.597461 28257 raft_consensus.cc:2781] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:37.597697 28257 raft_consensus.cc:686] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 1 LEADER]: Becoming Leader. State: Replica: e5b65a252de54eb0b33d6fd056f27c71, State: Running, Role: LEADER
I20260809 08:46:37.597811 28257 consensus_queue.cc:237] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b65a252de54eb0b33d6fd056f27c71" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41655 } } peers { permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 } } peers { permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 } }
I20260809 08:46:37.598742 27556 catalog_manager.cc:5495] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5b65a252de54eb0b33d6fd056f27c71 (127.31.63.65). New cstate: current_term: 1 leader_uuid: "e5b65a252de54eb0b33d6fd056f27c71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5b65a252de54eb0b33d6fd056f27c71" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 41655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 } health_report { overall_health: UNKNOWN } } }
W20260809 08:46:37.609009 28228 tablet.cc:2369] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:46:37.626384 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.623742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.31.63.65:41655)
0809 08:46:37.623814 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.623816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.623828 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:46:37.623878 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.623879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.623906 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.623990 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:46:37.624176 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.624674 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.625479 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.625753 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.625829 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.626026 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:46:37.626143 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:46:37.626249 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.626268 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.626292 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:46:37.628589 27955 raft_consensus.cc:1261] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527 [term 1 FOLLOWER]: Refusing update from remote peer e5b65a252de54eb0b33d6fd056f27c71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:37.628620 28164 raft_consensus.cc:1261] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289 [term 1 FOLLOWER]: Refusing update from remote peer e5b65a252de54eb0b33d6fd056f27c71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:37.629012 28257 consensus_queue.cc:1035] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50a6ad01baa64166a0eeefb1cfd3f527" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 40795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:37.629163 28263 consensus_queue.cc:1035] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015eb853dfcf460fb3c078c630d5f289" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 40947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:37.632632 28270 mvcc.cc:205] Tried to move back new op lower bound from 7316542249483317248 to 7316542249361354752. Current Snapshot: MvccSnapshot[applied={T|T < 7316542249483317248 or (T in {7316542249483317248})}]
I20260809 08:46:37.634428 27556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:37.634531 27556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:37.635392 27556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8497cc1e6ce412dbb610732b9d4b466: Sending DeleteTablet for 3 replicas of tablet fdcaacb43daa447fbf105ea044ceb8c7
I20260809 08:46:37.635764 28132 tablet_service.cc:1514] Processing DeleteTablet for tablet fdcaacb43daa447fbf105ea044ceb8c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:37 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:43684
I20260809 08:46:37.635814 27736 tablet_service.cc:1514] Processing DeleteTablet for tablet fdcaacb43daa447fbf105ea044ceb8c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:37 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:35702
I20260809 08:46:37.635900 27929 tablet_service.cc:1514] Processing DeleteTablet for tablet fdcaacb43daa447fbf105ea044ceb8c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:37 UTC) from {username='kudu', principal='kudu/127.31.63.126@KRBTEST.COM'} at 127.0.0.1:55018
I20260809 08:46:37.636068 28286 tablet_replica.cc:330] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71: stopping tablet replica
I20260809 08:46:37.636175 28286 raft_consensus.cc:2229] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:46:37.636193 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 27639
I20260809 08:46:37.636276 28286 raft_consensus.cc:2258] T fdcaacb43daa447fbf105ea044ceb8c7 P e5b65a252de54eb0b33d6fd056f27c71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:37.636441 28285 tablet_replica.cc:330] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: stopping tablet replica
I20260809 08:46:37.636523 28285 raft_consensus.cc:2229] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:37.640380 28285 raft_consensus.cc:2258] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:37.642779 28285 ts_tablet_manager.cc:1905] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:37.644420 28285 ts_tablet_manager.cc:1918] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:37.644526 28285 log.cc:1196] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/fdcaacb43daa447fbf105ea044ceb8c7
I20260809 08:46:37.644874 28287 tablet_replica.cc:330] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: stopping tablet replica
I20260809 08:46:37.644878 28285 ts_tablet_manager.cc:1939] T fdcaacb43daa447fbf105ea044ceb8c7 P 015eb853dfcf460fb3c078c630d5f289: Deleting consensus metadata
I20260809 08:46:37.645009 28287 raft_consensus.cc:2229] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:37.645152 28287 raft_consensus.cc:2258] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:37.645361 27540 catalog_manager.cc:4841] TS 015eb853dfcf460fb3c078c630d5f289 (127.31.63.67:40947): tablet fdcaacb43daa447fbf105ea044ceb8c7 (table test-table [id=cae3b763656744c3b3b3c38cdfaf4c66]) successfully deleted
I20260809 08:46:37.645684 28287 ts_tablet_manager.cc:1905] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:37.647011 28287 ts_tablet_manager.cc:1918] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:37.647082 28287 log.cc:1196] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/fdcaacb43daa447fbf105ea044ceb8c7
I20260809 08:46:37.647405 28287 ts_tablet_manager.cc:1939] T fdcaacb43daa447fbf105ea044ceb8c7 P 50a6ad01baa64166a0eeefb1cfd3f527: Deleting consensus metadata
I20260809 08:46:37.647840 27541 catalog_manager.cc:4841] TS 50a6ad01baa64166a0eeefb1cfd3f527 (127.31.63.66:40795): tablet fdcaacb43daa447fbf105ea044ceb8c7 (table test-table [id=cae3b763656744c3b3b3c38cdfaf4c66]) successfully deleted
W20260809 08:46:37.649924 27541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:41655 (error 108)
I20260809 08:46:37.650177 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 27835
I20260809 08:46:37.652325 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.650382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:41655 (local address 127.0.0.1:35722)
0809 08:46:37.652228 (+ 1846us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:37.652277 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:41655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1822}
W20260809 08:46:37.652498 27541 catalog_manager.cc:4584] TS e5b65a252de54eb0b33d6fd056f27c71 (127.31.63.65:41655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fdcaacb43daa447fbf105ea044ceb8c7: Network error: Client connection negotiation failed: client connection to 127.31.63.65:41655: connect: Connection refused (error 111)
I20260809 08:46:37.658003 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 28029
I20260809 08:46:37.670607 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 27520
2026-08-09T08:46:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2407 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T08:46:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:37Z Disabled control of system clock
I20260809 08:46:37.708746 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:43113
--webserver_interface=127.31.63.126
--webserver_port=0
--builtin_ntp_servers=127.31.63.84:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:43113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:46:37.792421 28294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:37.792641 28294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:37.792661 28294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:37.793926 28294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:37.793970 28294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:37.793982 28294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:37.793996 28294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:37.794008 28294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:37.795409 28294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:43113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:43113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:37.795668 28294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:37.795985 28294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:37.801097 28309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:37.801061 28302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:37.801101 28306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:37.801698 28294 server_base.cc:1034] running on GCE node
I20260809 08:46:37.801980 28294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:37.802309 28294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:37.803510 28294 hybrid_clock.cc:648] HybridClock initialized: now 1786265197803483 us; error 38 us; skew 500 ppm
I20260809 08:46:37.804942 28294 webserver.cc:460] Webserver started at http://127.31.63.126:45241/ using document root <none> and password file <none>
I20260809 08:46:37.805214 28294 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:37.805292 28294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:37.805457 28294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:37.806587 28294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "9a1a570caeee4da196abb250c3cab855"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "30911a52c993025360ce83aa7953f107"
server_key_iv: "46df5dcf0d155f91c85a6b2953546bce"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.806967 28294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "9a1a570caeee4da196abb250c3cab855"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "30911a52c993025360ce83aa7953f107"
server_key_iv: "46df5dcf0d155f91c85a6b2953546bce"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.808413 28294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:46:37.809319 28319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.809618 28294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:37.809753 28294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "9a1a570caeee4da196abb250c3cab855"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "30911a52c993025360ce83aa7953f107"
server_key_iv: "46df5dcf0d155f91c85a6b2953546bce"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.809859 28294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:37.819365 28294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:37.819729 28294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:37.819914 28294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:37.824652 28294 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:43113
I20260809 08:46:37.824702 28395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:43113 every 8 connection(s)
I20260809 08:46:37.825106 28294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:37.826247 28396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:37.828576 28396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855: Bootstrap starting.
I20260809 08:46:37.829205 28396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:37.829509 28396 log.cc:824] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:37.830323 28396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855: No bootstrap required, opened a new log
I20260809 08:46:37.831925 28396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1a570caeee4da196abb250c3cab855" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 43113 } }
I20260809 08:46:37.832065 28396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:37.832087 28396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a1a570caeee4da196abb250c3cab855, State: Initialized, Role: FOLLOWER
I20260809 08:46:37.832270 28396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1a570caeee4da196abb250c3cab855" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 43113 } }
I20260809 08:46:37.832360 28396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:37.832402 28396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:37.832453 28396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:37.832968 28396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1a570caeee4da196abb250c3cab855" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 43113 } }
I20260809 08:46:37.833053 28396 leader_election.cc:302] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a1a570caeee4da196abb250c3cab855; no voters:
I20260809 08:46:37.833236 28396 leader_election.cc:288] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:37.833357 28402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:37.833561 28402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [term 1 LEADER]: Becoming Leader. State: Replica: 9a1a570caeee4da196abb250c3cab855, State: Running, Role: LEADER
I20260809 08:46:37.833650 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 28294
I20260809 08:46:37.833690 28402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1a570caeee4da196abb250c3cab855" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 43113 } }
I20260809 08:46:37.833767 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:37.833992 31997 external_mini_cluster.cc:1428] Setting key 1abb3078e3b928794ae4a9805379db2d
I20260809 08:46:37.834051 28403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a1a570caeee4da196abb250c3cab855" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1a570caeee4da196abb250c3cab855" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 43113 } } }
I20260809 08:46:37.834136 28403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:37.834170 28396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:37.834414 28405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a1a570caeee4da196abb250c3cab855. Latest consensus state: current_term: 1 leader_uuid: "9a1a570caeee4da196abb250c3cab855" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1a570caeee4da196abb250c3cab855" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 43113 } } }
I20260809 08:46:37.834479 28405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:37.835048 28407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:37.835588 28407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:37.837350 28407 catalog_manager.cc:1347] Generated new cluster ID: fb24e640f3f1477f91b81967a8fe89de
I20260809 08:46:37.837404 28407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:37.840672 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.834731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.31.63.126:43113)
0809 08:46:37.835137 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.835147 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.835185 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:46:37.835303 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.835311 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.835385 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.835614 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:37.835857 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.837018 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.838222 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.838872 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.839028 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.839867 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:37.839872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:37.839872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:37.840128 (+ 256us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:37.840176 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:37.840193 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:37.840283 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.840349 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.840435 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":87,"threads_started":1}
I20260809 08:46:37.857610 28407 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:37.858253 28407 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:37.862699 28407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855: Generated new TSK 0
I20260809 08:46:37.862908 28407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:37.868162 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.65:0
--local_ip_for_outbound_sockets=127.31.63.65
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:43113
--builtin_ntp_servers=127.31.63.84:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:46:37.952363 28432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:37.952574 28432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:37.952595 28432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:37.953799 28432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:37.953841 28432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:37.953866 28432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.65
I20260809 08:46:37.955231 28432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.63.65
--webserver_port=0
--tserver_master_addrs=127.31.63.126:43113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.65
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:37.955503 28432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:37.955843 28432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:37.961884 28441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:37.961906 28442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:37.962287 28432 server_base.cc:1034] running on GCE node
W20260809 08:46:37.961935 28445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:37.962536 28432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:37.962808 28432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:37.963975 28432 hybrid_clock.cc:648] HybridClock initialized: now 1786265197963962 us; error 32 us; skew 500 ppm
I20260809 08:46:37.965320 28432 webserver.cc:460] Webserver started at http://127.31.63.65:44789/ using document root <none> and password file <none>
I20260809 08:46:37.965641 28432 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:37.965718 28432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:37.965859 28432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:37.966930 28432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/data/instance:
uuid: "a75f29e1a0a548d9a4de75212ea92f29"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "3adbf33a37f81040c271803c6e0ee443"
server_key_iv: "864b8bd769328c6dde0a5249d998fb45"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.967314 28432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance:
uuid: "a75f29e1a0a548d9a4de75212ea92f29"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "3adbf33a37f81040c271803c6e0ee443"
server_key_iv: "864b8bd769328c6dde0a5249d998fb45"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.968722 28432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:37.969576 28453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.969762 28432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:37.969883 28432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/wal
uuid: "a75f29e1a0a548d9a4de75212ea92f29"
format_stamp: "Formatted at 2026-08-09 08:46:37 on dist-test-slave-gkw7"
server_key: "3adbf33a37f81040c271803c6e0ee443"
server_key_iv: "864b8bd769328c6dde0a5249d998fb45"
server_key_version: "encryptionkey@0"
I20260809 08:46:37.969969 28432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:37.983987 28432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:37.984323 28432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:37.984478 28432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:37.984724 28432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:37.985064 28432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:37.985144 28432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.985217 28432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:37.985250 28432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:37.991827 28432 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.65:42875
I20260809 08:46:37.991884 28596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.65:42875 every 8 connection(s)
I20260809 08:46:37.992266 28432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/data/info.pb
I20260809 08:46:37.994319 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 28432
I20260809 08:46:37.994455 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-0/wal/instance
I20260809 08:46:37.994683 31997 external_mini_cluster.cc:1428] Setting key 10f1d9101dd23a6ae85baa164424ce69
I20260809 08:46:37.995709 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.66:0
--local_ip_for_outbound_sockets=127.31.63.66
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:43113
--builtin_ntp_servers=127.31.63.84:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 08:46:37.998121 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.993095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.65:38479 (local address 127.31.63.126:43113)
0809 08:46:37.993162 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:37.993164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:37.993403 (+ 239us) server_negotiation.cc:407] Connection header received
0809 08:46:37.993927 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:37.993938 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:37.993952 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:37.994013 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:37.994794 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.995427 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.996540 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.996756 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.996837 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.997818 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:37.997820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:37.997820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:37.997827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:37.997843 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:37.997848 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:37.997909 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:37.998016 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 08:46:37.998058 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:46:37.998392 28603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:37.993069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:43113 (local address 127.31.63.65:38479)
0809 08:46:37.993274 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:37.993308 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:37.993814 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:37.994067 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:37.994071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:37.994079 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:37.994693 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.994699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.995513 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.995514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.996431 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:37.996434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:37.996846 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:37.996848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:37.996991 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:37.997624 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:46:37.997625 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:37.997668 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:46:37.997702 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:37.997925 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:37.997931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:37.997934 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:46:37.998008 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 08:46:37.998132 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260809 08:46:37.998956 28599 heartbeater.cc:347] Connected to a master server at 127.31.63.126:43113
I20260809 08:46:37.999056 28599 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:37.999244 28599 heartbeater.cc:511] Master 127.31.63.126:43113 requested a full tablet report, sending...
I20260809 08:46:37.999923 28346 ts_manager.cc:194] Registered new tserver with Master: a75f29e1a0a548d9a4de75212ea92f29 (127.31.63.65:42875)
I20260809 08:46:38.000900 28346 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.65:38479
W20260809 08:46:38.083519 28605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:38.083746 28605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:38.083768 28605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:38.084844 28605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:38.084877 28605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:38.084903 28605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.66
I20260809 08:46:38.086254 28605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.63.66
--webserver_port=0
--tserver_master_addrs=127.31.63.126:43113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.66
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:38.086450 28605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:38.086707 28605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:38.091987 28612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:38.091995 28615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:38.092005 28611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:38.092312 28605 server_base.cc:1034] running on GCE node
I20260809 08:46:38.092528 28605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:38.092716 28605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:38.093855 28605 hybrid_clock.cc:648] HybridClock initialized: now 1786265198093847 us; error 27 us; skew 500 ppm
I20260809 08:46:38.095191 28605 webserver.cc:460] Webserver started at http://127.31.63.66:44611/ using document root <none> and password file <none>
I20260809 08:46:38.095458 28605 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:38.095533 28605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:38.095693 28605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:38.096781 28605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/data/instance:
uuid: "87d9733c8e4e40d5afe67cd577132871"
format_stamp: "Formatted at 2026-08-09 08:46:38 on dist-test-slave-gkw7"
server_key: "076e67590987643da609dc03271b5c5b"
server_key_iv: "d7b7f086fe99a54c12bdddfd71229437"
server_key_version: "encryptionkey@0"
I20260809 08:46:38.097143 28605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance:
uuid: "87d9733c8e4e40d5afe67cd577132871"
format_stamp: "Formatted at 2026-08-09 08:46:38 on dist-test-slave-gkw7"
server_key: "076e67590987643da609dc03271b5c5b"
server_key_iv: "d7b7f086fe99a54c12bdddfd71229437"
server_key_version: "encryptionkey@0"
I20260809 08:46:38.098592 28605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:38.099440 28624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:38.099627 28605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:38.099736 28605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/wal
uuid: "87d9733c8e4e40d5afe67cd577132871"
format_stamp: "Formatted at 2026-08-09 08:46:38 on dist-test-slave-gkw7"
server_key: "076e67590987643da609dc03271b5c5b"
server_key_iv: "d7b7f086fe99a54c12bdddfd71229437"
server_key_version: "encryptionkey@0"
I20260809 08:46:38.099823 28605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:38.106823 28605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:38.107136 28605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:38.107291 28605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:38.107518 28605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:38.107873 28605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:38.107931 28605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:38.107995 28605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:38.108039 28605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:38.114732 28605 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.66:41195
I20260809 08:46:38.114784 28785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.66:41195 every 8 connection(s)
I20260809 08:46:38.115163 28605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/data/info.pb
I20260809 08:46:38.120535 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.115999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:57061 (local address 127.31.63.126:43113)
0809 08:46:38.116087 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:38.116090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:38.116356 (+ 266us) server_negotiation.cc:407] Connection header received
0809 08:46:38.116860 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:38.116861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:38.116873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:38.116933 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:38.117682 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.118210 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.119197 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.119383 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.119437 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.120328 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:38.120329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:38.120329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:38.120334 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:38.120344 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:38.120347 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:38.120384 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:38.120447 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 08:46:38.120472 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 08:46:38.120782 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.115965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:43113 (local address 127.31.63.66:57061)
0809 08:46:38.116187 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:38.116222 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:38.116741 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:38.116991 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:38.116994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:38.117003 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:38.117520 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.117580 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.118316 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.118317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.119119 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.119121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.119462 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.119463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.119580 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.120181 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:46:38.120181 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:38.120212 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:46:38.120236 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:38.120386 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:38.120388 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:38.120391 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:46:38.120438 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 08:46:38.120522 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
I20260809 08:46:38.121364 28788 heartbeater.cc:347] Connected to a master server at 127.31.63.126:43113
I20260809 08:46:38.121476 28788 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:38.121697 28788 heartbeater.cc:511] Master 127.31.63.126:43113 requested a full tablet report, sending...
I20260809 08:46:38.122109 28346 ts_manager.cc:194] Registered new tserver with Master: 87d9733c8e4e40d5afe67cd577132871 (127.31.63.66:41195)
I20260809 08:46:38.122499 28346 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.66:57061
I20260809 08:46:38.122761 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 28605
I20260809 08:46:38.122889 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/wal/instance
I20260809 08:46:38.123106 31997 external_mini_cluster.cc:1428] Setting key 2d444d7323ad4e178c23f6290d317671
I20260809 08:46:38.124037 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.63.67:0
--local_ip_for_outbound_sockets=127.31.63.67
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:43113
--builtin_ntp_servers=127.31.63.84:35151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:46:38.211931 28793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:38.212155 28793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:38.212180 28793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:38.213408 28793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:38.213451 28793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:38.213508 28793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.63.67
I20260809 08:46:38.214941 28793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.63.67
--webserver_port=0
--tserver_master_addrs=127.31.63.126:43113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.63.67
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:38.215150 28793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:38.215437 28793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:38.220824 28800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:38.220852 28804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:38.220865 28793 server_base.cc:1034] running on GCE node
W20260809 08:46:38.220793 28801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:38.221298 28793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:38.221645 28793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:38.222831 28793 hybrid_clock.cc:648] HybridClock initialized: now 1786265198222826 us; error 37 us; skew 500 ppm
I20260809 08:46:38.224231 28793 webserver.cc:460] Webserver started at http://127.31.63.67:41815/ using document root <none> and password file <none>
I20260809 08:46:38.224498 28793 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:38.224577 28793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:38.224742 28793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:38.225970 28793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/data/instance:
uuid: "bd71e275fc6f40afa3ad9900ac4f5066"
format_stamp: "Formatted at 2026-08-09 08:46:38 on dist-test-slave-gkw7"
server_key: "2c1e60daa0591077910549dfbebce965"
server_key_iv: "a71820a065055a4fd66e4beb61c8cf20"
server_key_version: "encryptionkey@0"
I20260809 08:46:38.226341 28793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance:
uuid: "bd71e275fc6f40afa3ad9900ac4f5066"
format_stamp: "Formatted at 2026-08-09 08:46:38 on dist-test-slave-gkw7"
server_key: "2c1e60daa0591077910549dfbebce965"
server_key_iv: "a71820a065055a4fd66e4beb61c8cf20"
server_key_version: "encryptionkey@0"
I20260809 08:46:38.227728 28793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:46:38.228648 28813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:38.228835 28793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:38.228947 28793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/wal
uuid: "bd71e275fc6f40afa3ad9900ac4f5066"
format_stamp: "Formatted at 2026-08-09 08:46:38 on dist-test-slave-gkw7"
server_key: "2c1e60daa0591077910549dfbebce965"
server_key_iv: "a71820a065055a4fd66e4beb61c8cf20"
server_key_version: "encryptionkey@0"
I20260809 08:46:38.229025 28793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:38.259054 28793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:38.259446 28793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:38.259622 28793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:38.259902 28793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:46:38.260303 28793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:46:38.260361 28793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:38.260403 28793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:46:38.260445 28793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:38.266942 28793 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.67:33341
I20260809 08:46:38.266995 28978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.67:33341 every 8 connection(s)
I20260809 08:46:38.267365 28793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/data/info.pb
I20260809 08:46:38.270465 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 28793
I20260809 08:46:38.270591 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/wal/instance
I20260809 08:46:38.270810 31997 external_mini_cluster.cc:1428] Setting key 06344af08a733a5dbb2f63f59496c34f
I20260809 08:46:38.273516 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.268167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.67:40967 (local address 127.31.63.126:43113)
0809 08:46:38.268242 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:38.268244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:38.268530 (+ 286us) server_negotiation.cc:407] Connection header received
0809 08:46:38.269039 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:38.269040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:38.269052 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:38.269109 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:38.269882 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.270396 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.271718 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.272069 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.272136 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.273270 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:38.273272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:38.273272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:38.273277 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:38.273291 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:38.273296 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:38.273344 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:38.273429 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 08:46:38.273455 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:46:38.273780 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.268159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:43113 (local address 127.31.63.67:40967)
0809 08:46:38.268396 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:38.268432 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:38.268952 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:38.269184 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:38.269187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:38.269195 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:46:38.269805 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.269811 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.270496 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.270497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.271609 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.271612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.272149 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.272151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.272428 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.273130 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:46:38.273130 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:46:38.273161 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:46:38.273184 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:46:38.273359 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:46:38.273363 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:46:38.273365 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:46:38.273414 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 08:46:38.273500 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":118,"threads_started":1}
I20260809 08:46:38.274312 28979 heartbeater.cc:347] Connected to a master server at 127.31.63.126:43113
I20260809 08:46:38.274397 28979 heartbeater.cc:464] Registering TS with master...
I20260809 08:46:38.274605 28979 heartbeater.cc:511] Master 127.31.63.126:43113 requested a full tablet report, sending...
I20260809 08:46:38.274995 28346 ts_manager.cc:194] Registered new tserver with Master: bd71e275fc6f40afa3ad9900ac4f5066 (127.31.63.67:33341)
I20260809 08:46:38.275341 28346 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.63.67:40967
I20260809 08:46:38.283041 31997 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:46:38.287633 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.284625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49666 (local address 127.31.63.126:43113)
0809 08:46:38.284739 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:38.284742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:38.284834 (+ 92us) server_negotiation.cc:407] Connection header received
0809 08:46:38.284899 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:38.284900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:38.284914 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:38.284963 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:38.285178 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.285734 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.286510 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.286718 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.286785 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.287365 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:38.287366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:38.287367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:38.287371 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:38.287385 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:38.287388 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:38.287435 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:38.287471 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 08:46:38.287496 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 08:46:38.289247 28346 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:46:38.290920 28346 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:46:38.297609 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.293317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.66:41195 (local address 127.0.0.1:40346)
0809 08:46:38.293760 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:38.293770 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:38.293810 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:38.294094 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:38.294098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:38.294101 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:38.294176 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.294178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.295124 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.295125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.296369 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.296372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.297259 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.297261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.297350 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.297367 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 08:46:38.297437 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:46:38.297492 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":93,"threads_started":1}
I20260809 08:46:38.297734 28991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.293226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:42875 (local address 127.0.0.1:51674)
0809 08:46:38.293424 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:38.293467 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:38.293577 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:38.294130 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:38.294131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:38.294132 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:38.294186 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.294188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.295538 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.295539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.296504 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.296507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.297399 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.297401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.297468 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.297472 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:38.297570 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 08:46:38.297613 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":12,"thread_start_us":84,"threads_started":1}
I20260809 08:46:38.297749 28993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.293338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51674 (local address 127.31.63.65:42875)
0809 08:46:38.293961 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:38.293964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:38.293975 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:46:38.294015 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:38.294017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:38.294031 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:38.294124 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:38.294277 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.295409 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.296619 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.297273 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.297388 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.297482 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:38.297599 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 08:46:38.297644 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":92,"threads_started":1}
I20260809 08:46:38.297981 28997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.293599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40346 (local address 127.31.63.66:41195)
0809 08:46:38.293802 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:38.293807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:38.293820 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:46:38.293895 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:38.293897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:38.293914 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:38.293997 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:38.294270 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.295000 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.296471 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.297164 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.297239 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.297328 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:38.297843 (+ 515us) server_negotiation.cc:299] Negotiation successful
0809 08:46:38.297897 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":112,"threads_started":1}
I20260809 08:46:38.298061 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.293304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:33341 (local address 127.0.0.1:60858)
0809 08:46:38.293619 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:38.293629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:38.293674 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:38.294125 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:38.294126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:38.294128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:38.294187 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.294188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.295527 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.295528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.296799 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.296803 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.297820 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.297821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.297894 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.297898 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:38.297954 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 08:46:38.298002 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":66,"thread_start_us":130,"threads_started":1}
I20260809 08:46:38.298355 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.293599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60858 (local address 127.31.63.67:33341)
0809 08:46:38.293893 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:38.293898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:38.293908 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:46:38.293963 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:38.293965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:38.293982 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:38.294060 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:38.294268 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.295409 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.296897 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.297631 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.297703 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.297783 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:38.298212 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 08:46:38.298254 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":112,"threads_started":1}
I20260809 08:46:38.298522 28514 tablet_service.cc:1467] Processing CreateTablet for tablet b30233e77ac841bfaa8d2ea8ac28b99b (DEFAULT_TABLE table=test-table [id=e91239ee90ea4a6abbfe63a369199868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:38.298533 28687 tablet_service.cc:1467] Processing CreateTablet for tablet b30233e77ac841bfaa8d2ea8ac28b99b (DEFAULT_TABLE table=test-table [id=e91239ee90ea4a6abbfe63a369199868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:38.298918 28687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b30233e77ac841bfaa8d2ea8ac28b99b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:38.298918 28514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b30233e77ac841bfaa8d2ea8ac28b99b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:38.298977 28881 tablet_service.cc:1467] Processing CreateTablet for tablet b30233e77ac841bfaa8d2ea8ac28b99b (DEFAULT_TABLE table=test-table [id=e91239ee90ea4a6abbfe63a369199868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:46:38.299365 28881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b30233e77ac841bfaa8d2ea8ac28b99b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:38.301999 29000 tablet_bootstrap.cc:492] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: Bootstrap starting.
I20260809 08:46:38.302472 29001 tablet_bootstrap.cc:492] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: Bootstrap starting.
I20260809 08:46:38.302882 29000 tablet_bootstrap.cc:654] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:38.303158 29001 tablet_bootstrap.cc:654] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:38.303257 29000 log.cc:824] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:38.303500 29001 log.cc:824] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:38.303845 29003 tablet_bootstrap.cc:492] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29: Bootstrap starting.
I20260809 08:46:38.304267 29000 tablet_bootstrap.cc:492] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: No bootstrap required, opened a new log
I20260809 08:46:38.304286 29001 tablet_bootstrap.cc:492] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: No bootstrap required, opened a new log
I20260809 08:46:38.304354 29001 ts_tablet_manager.cc:1397] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:46:38.304358 29000 ts_tablet_manager.cc:1397] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:46:38.304569 29003 tablet_bootstrap.cc:654] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:38.304915 29003 log.cc:824] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:38.305948 29003 tablet_bootstrap.cc:492] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29: No bootstrap required, opened a new log
I20260809 08:46:38.306016 29003 ts_tablet_manager.cc:1397] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:46:38.306628 29001 raft_consensus.cc:348] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 } } peers { permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 } } peers { permanent_uuid: "87d9733c8e4e40d5afe67cd577132871" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41195 } }
I20260809 08:46:38.306761 29001 raft_consensus.cc:374] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:38.306780 29001 raft_consensus.cc:729] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87d9733c8e4e40d5afe67cd577132871, State: Initialized, Role: FOLLOWER
I20260809 08:46:38.306905 29001 consensus_queue.cc:260] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 } } peers { permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 } } peers { permanent_uuid: "87d9733c8e4e40d5afe67cd577132871" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41195 } }
I20260809 08:46:38.307205 28788 heartbeater.cc:503] Master 127.31.63.126:43113 was elected leader, sending a full tablet report...
I20260809 08:46:38.307130 29000 raft_consensus.cc:348] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 } } peers { permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 } } peers { permanent_uuid: "87d9733c8e4e40d5afe67cd577132871" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41195 } }
I20260809 08:46:38.307283 29000 raft_consensus.cc:374] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:38.307319 29000 raft_consensus.cc:729] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd71e275fc6f40afa3ad9900ac4f5066, State: Initialized, Role: FOLLOWER
I20260809 08:46:38.307457 29001 ts_tablet_manager.cc:1428] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:46:38.307427 29000 consensus_queue.cc:260] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 } } peers { permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 } } peers { permanent_uuid: "87d9733c8e4e40d5afe67cd577132871" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41195 } }
I20260809 08:46:38.307695 29000 ts_tablet_manager.cc:1428] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 08:46:38.307775 28979 heartbeater.cc:503] Master 127.31.63.126:43113 was elected leader, sending a full tablet report...
I20260809 08:46:38.308650 29003 raft_consensus.cc:348] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 } } peers { permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 } } peers { permanent_uuid: "87d9733c8e4e40d5afe67cd577132871" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41195 } }
I20260809 08:46:38.308792 29003 raft_consensus.cc:374] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:38.308830 29003 raft_consensus.cc:729] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75f29e1a0a548d9a4de75212ea92f29, State: Initialized, Role: FOLLOWER
I20260809 08:46:38.308931 29003 consensus_queue.cc:260] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 } } peers { permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 } } peers { permanent_uuid: "87d9733c8e4e40d5afe67cd577132871" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41195 } }
I20260809 08:46:38.309149 29003 ts_tablet_manager.cc:1428] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:46:38.309264 28599 heartbeater.cc:503] Master 127.31.63.126:43113 was elected leader, sending a full tablet report...
I20260809 08:46:38.356303 29011 raft_consensus.cc:482] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:46:38.356460 29011 raft_consensus.cc:504] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 } } peers { permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 } } peers { permanent_uuid: "87d9733c8e4e40d5afe67cd577132871" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41195 } }
I20260809 08:46:38.356940 29011 leader_election.cc:288] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a75f29e1a0a548d9a4de75212ea92f29 (127.31.63.65:42875), bd71e275fc6f40afa3ad9900ac4f5066 (127.31.63.67:33341)
I20260809 08:46:38.360189 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.357156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.67:33341 (local address 127.31.63.66:60441)
0809 08:46:38.357232 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:38.357250 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:38.357299 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:38.357504 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:38.357506 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:38.357508 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:38.357631 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.357634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.358362 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.358364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.359250 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.359253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.359862 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.359863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.359939 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.359943 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:38.360015 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:46:38.360105 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":22}
I20260809 08:46:38.360201 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.357248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:60441 (local address 127.31.63.67:33341)
0809 08:46:38.357323 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:38.357326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:38.357366 (+ 40us) server_negotiation.cc:407] Connection header received
0809 08:46:38.357408 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:38.357409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:38.357418 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:38.357495 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:38.357735 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.358267 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.359342 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.359771 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.359828 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.359836 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:38.360031 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 08:46:38.360113 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:46:38.360308 29014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.357156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:42875 (local address 127.31.63.66:33365)
0809 08:46:38.357476 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:38.357485 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:46:38.357584 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:46:38.357738 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:46:38.357739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:46:38.357741 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:46:38.357810 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.357811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.358550 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.358551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.359402 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:46:38.359404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.360036 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.360037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:46:38.360115 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.360119 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:46:38.360179 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:46:38.360217 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":140,"thread_start_us":137,"threads_started":1}
I20260809 08:46:38.360339 28993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.357288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.63.66:33365 (local address 127.31.63.65:42875)
0809 08:46:38.357339 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:38.357341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:38.357593 (+ 252us) server_negotiation.cc:407] Connection header received
0809 08:46:38.357636 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:38.357638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:38.357653 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:38.357712 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:46:38.357890 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.358459 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.359500 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.359944 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.360026 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.360034 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:38.360203 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 08:46:38.360255 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:46:38.360608 28912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b30233e77ac841bfaa8d2ea8ac28b99b" candidate_uuid: "87d9733c8e4e40d5afe67cd577132871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" is_pre_election: true
I20260809 08:46:38.360639 28537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b30233e77ac841bfaa8d2ea8ac28b99b" candidate_uuid: "87d9733c8e4e40d5afe67cd577132871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75f29e1a0a548d9a4de75212ea92f29" is_pre_election: true
I20260809 08:46:38.360787 28537 raft_consensus.cc:2454] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87d9733c8e4e40d5afe67cd577132871 in term 0.
I20260809 08:46:38.360788 28912 raft_consensus.cc:2454] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87d9733c8e4e40d5afe67cd577132871 in term 0.
I20260809 08:46:38.361096 28625 leader_election.cc:302] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87d9733c8e4e40d5afe67cd577132871, a75f29e1a0a548d9a4de75212ea92f29; no voters:
I20260809 08:46:38.361282 29011 raft_consensus.cc:2781] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:46:38.361353 29011 raft_consensus.cc:482] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:46:38.361411 29011 raft_consensus.cc:3037] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:38.362313 29011 raft_consensus.cc:504] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 } } peers { permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 } } peers { permanent_uuid: "87d9733c8e4e40d5afe67cd577132871" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41195 } }
I20260809 08:46:38.362557 29011 leader_election.cc:288] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [CANDIDATE]: Term 1 election: Requested vote from peers a75f29e1a0a548d9a4de75212ea92f29 (127.31.63.65:42875), bd71e275fc6f40afa3ad9900ac4f5066 (127.31.63.67:33341)
I20260809 08:46:38.362641 28537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b30233e77ac841bfaa8d2ea8ac28b99b" candidate_uuid: "87d9733c8e4e40d5afe67cd577132871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a75f29e1a0a548d9a4de75212ea92f29"
I20260809 08:46:38.362659 28912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b30233e77ac841bfaa8d2ea8ac28b99b" candidate_uuid: "87d9733c8e4e40d5afe67cd577132871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd71e275fc6f40afa3ad9900ac4f5066"
I20260809 08:46:38.362756 28912 raft_consensus.cc:3037] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:38.362732 28537 raft_consensus.cc:3037] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:38.363592 28912 raft_consensus.cc:2454] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87d9733c8e4e40d5afe67cd577132871 in term 1.
I20260809 08:46:38.363708 28537 raft_consensus.cc:2454] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87d9733c8e4e40d5afe67cd577132871 in term 1.
I20260809 08:46:38.363786 28628 leader_election.cc:302] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87d9733c8e4e40d5afe67cd577132871, bd71e275fc6f40afa3ad9900ac4f5066; no voters:
I20260809 08:46:38.363904 29011 raft_consensus.cc:2781] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:38.364076 29011 raft_consensus.cc:686] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 1 LEADER]: Becoming Leader. State: Replica: 87d9733c8e4e40d5afe67cd577132871, State: Running, Role: LEADER
I20260809 08:46:38.364188 29011 consensus_queue.cc:237] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 } } peers { permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 } } peers { permanent_uuid: "87d9733c8e4e40d5afe67cd577132871" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41195 } }
I20260809 08:46:38.365094 28345 catalog_manager.cc:5495] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87d9733c8e4e40d5afe67cd577132871 (127.31.63.66). New cstate: current_term: 1 leader_uuid: "87d9733c8e4e40d5afe67cd577132871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87d9733c8e4e40d5afe67cd577132871" member_type: VOTER last_known_addr { host: "127.31.63.66" port: 41195 } health_report { overall_health: HEALTHY } } }
W20260809 08:46:38.365728 28790 tablet.cc:2369] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:46:38.370754 28997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.368561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40352 (local address 127.31.63.66:41195)
0809 08:46:38.368628 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:38.368630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:38.368638 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:46:38.368675 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:38.368676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:38.368692 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:38.368749 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:46:38.368855 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.369348 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.370119 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:38.370300 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:38.370356 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:38.370490 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:46:38.370579 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:46:38.370642 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:38.370676 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 08:46:38.370695 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 08:46:38.372469 28912 raft_consensus.cc:1261] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066 [term 1 FOLLOWER]: Refusing update from remote peer 87d9733c8e4e40d5afe67cd577132871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:38.372488 28537 raft_consensus.cc:1261] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29 [term 1 FOLLOWER]: Refusing update from remote peer 87d9733c8e4e40d5afe67cd577132871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:46:38.372813 29015 consensus_queue.cc:1035] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a75f29e1a0a548d9a4de75212ea92f29" member_type: VOTER last_known_addr { host: "127.31.63.65" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:38.372958 29011 consensus_queue.cc:1035] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd71e275fc6f40afa3ad9900ac4f5066" member_type: VOTER last_known_addr { host: "127.31.63.67" port: 33341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:46:38.375422 29018 mvcc.cc:205] Tried to move back new op lower bound from 7316542252530970624 to 7316542252500357120. Current Snapshot: MvccSnapshot[applied={T|T < 7316542252530970624}]
I20260809 08:46:38.375648 29021 mvcc.cc:205] Tried to move back new op lower bound from 7316542252530970624 to 7316542252500357120. Current Snapshot: MvccSnapshot[applied={T|T < 7316542252530970624}]
I20260809 08:46:38.376749 29020 mvcc.cc:205] Tried to move back new op lower bound from 7316542252530970624 to 7316542252500357120. Current Snapshot: MvccSnapshot[applied={T|T < 7316542252530970624}]
I20260809 08:46:38.378085 28345 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:38.378177 28345 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:46:38.378893 28345 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a1a570caeee4da196abb250c3cab855: Sending DeleteTablet for 3 replicas of tablet b30233e77ac841bfaa8d2ea8ac28b99b
I20260809 08:46:38.379294 28514 tablet_service.cc:1514] Processing DeleteTablet for tablet b30233e77ac841bfaa8d2ea8ac28b99b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:38 UTC) from {username='slave'} at 127.0.0.1:51674
I20260809 08:46:38.379303 28687 tablet_service.cc:1514] Processing DeleteTablet for tablet b30233e77ac841bfaa8d2ea8ac28b99b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:38 UTC) from {username='slave'} at 127.0.0.1:40346
I20260809 08:46:38.379345 28881 tablet_service.cc:1514] Processing DeleteTablet for tablet b30233e77ac841bfaa8d2ea8ac28b99b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:46:38 UTC) from {username='slave'} at 127.0.0.1:60858
I20260809 08:46:38.379542 29037 tablet_replica.cc:330] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: stopping tablet replica
I20260809 08:46:38.379560 29038 tablet_replica.cc:330] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29: stopping tablet replica
I20260809 08:46:38.379652 29037 raft_consensus.cc:2229] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:46:38.379755 29037 raft_consensus.cc:2258] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:38.379835 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 28432
I20260809 08:46:38.379834 29038 raft_consensus.cc:2229] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:38.380002 29038 raft_consensus.cc:2258] T b30233e77ac841bfaa8d2ea8ac28b99b P a75f29e1a0a548d9a4de75212ea92f29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:38.380446 29037 ts_tablet_manager.cc:1905] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:38.382555 29037 ts_tablet_manager.cc:1918] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:38.382624 29037 log.cc:1196] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-1/wal/wals/b30233e77ac841bfaa8d2ea8ac28b99b
I20260809 08:46:38.382900 29037 ts_tablet_manager.cc:1939] T b30233e77ac841bfaa8d2ea8ac28b99b P 87d9733c8e4e40d5afe67cd577132871: Deleting consensus metadata
I20260809 08:46:38.384236 28321 catalog_manager.cc:4841] TS 87d9733c8e4e40d5afe67cd577132871 (127.31.63.66:41195): tablet b30233e77ac841bfaa8d2ea8ac28b99b (table test-table [id=e91239ee90ea4a6abbfe63a369199868]) successfully deleted
I20260809 08:46:38.389642 29034 tablet_replica.cc:330] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: stopping tablet replica
I20260809 08:46:38.389783 29034 raft_consensus.cc:2229] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:46:38.389904 29034 raft_consensus.cc:2258] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:46:38.392010 29034 ts_tablet_manager.cc:1905] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:46:38.393718 29034 ts_tablet_manager.cc:1918] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:46:38.393798 29034 log.cc:1196] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: Deleting WAL directory at /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786264918773806-31997-0/minicluster-data/ts-2/wal/wals/b30233e77ac841bfaa8d2ea8ac28b99b
I20260809 08:46:38.394052 29034 ts_tablet_manager.cc:1939] T b30233e77ac841bfaa8d2ea8ac28b99b P bd71e275fc6f40afa3ad9900ac4f5066: Deleting consensus metadata
I20260809 08:46:38.394490 28323 catalog_manager.cc:4841] TS bd71e275fc6f40afa3ad9900ac4f5066 (127.31.63.67:33341): tablet b30233e77ac841bfaa8d2ea8ac28b99b (table test-table [id=e91239ee90ea4a6abbfe63a369199868]) successfully deleted
W20260809 08:46:38.398113 28320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.63.65:42875 (error 108)
I20260809 08:46:38.398348 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 28605
I20260809 08:46:38.401726 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:38.401584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.65:42875 (local address 127.0.0.1:51682)
0809 08:46:38.401639 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:46:38.401684 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.65:42875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 08:46:38.402246 28320 catalog_manager.cc:4584] TS a75f29e1a0a548d9a4de75212ea92f29 (127.31.63.65:42875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b30233e77ac841bfaa8d2ea8ac28b99b: Network error: Client connection negotiation failed: client connection to 127.31.63.65:42875: connect: Connection refused (error 111)
I20260809 08:46:38.410916 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 28793
I20260809 08:46:38.427950 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 28294
2026-08-09T08:46:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (753 ms)
[----------] 6 tests from EncryptionPolicyTest (9498 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:38 dist-test-slave-gkw7 krb5kdc[29042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:38 dist-test-slave-gkw7 krb5kdc[29042](info): set up 2 sockets
Aug 09 08:46:38 dist-test-slave-gkw7 krb5kdc[29042](info): commencing operation
krb5kdc: starting...
W20260809 08:46:39.658615 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:39 dist-test-slave-gkw7 krb5kdc[29042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:39.788307 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45053
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:39.872301 29058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:39.872563 29058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:39.872598 29058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:39.873762 29058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:39.873850 29058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:39.873876 29058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:39.873896 29058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:39.873926 29058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:39.875430 29058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45053
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:39.875710 29058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:39.875957 29058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:39.881282 29065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:39.881297 29066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:39.881573 29058 server_base.cc:1034] running on GCE node
W20260809 08:46:39.881300 29068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:39.881939 29058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:39.882315 29058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:39.883514 29058 hybrid_clock.cc:648] HybridClock initialized: now 1786265199883495 us; error 55 us; skew 500 ppm
Aug 09 08:46:39 dist-test-slave-gkw7 krb5kdc[29042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:39.886729 29058 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:39.887195 29058 webserver.cc:460] Webserver started at http://127.0.0.1:45821/ using document root <none> and password file <none>
I20260809 08:46:39.887472 29058 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:39.887521 29058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:39.887691 29058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:39.888811 29058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "01810066a66647ad95ab2232c1980772"
format_stamp: "Formatted at 2026-08-09 08:46:39 on dist-test-slave-gkw7"
server_key: "9f818f08feb084e3b403f5f2ff6ccf45"
server_key_iv: "6b2df27356474efddd3e402bd7e074c6"
server_key_version: "encryptionkey@0"
I20260809 08:46:39.889190 29058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "01810066a66647ad95ab2232c1980772"
format_stamp: "Formatted at 2026-08-09 08:46:39 on dist-test-slave-gkw7"
server_key: "9f818f08feb084e3b403f5f2ff6ccf45"
server_key_iv: "6b2df27356474efddd3e402bd7e074c6"
server_key_version: "encryptionkey@0"
I20260809 08:46:39.890647 29058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:39.891393 29081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:39.891573 29058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:46:39.891673 29058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "01810066a66647ad95ab2232c1980772"
format_stamp: "Formatted at 2026-08-09 08:46:39 on dist-test-slave-gkw7"
server_key: "9f818f08feb084e3b403f5f2ff6ccf45"
server_key_iv: "6b2df27356474efddd3e402bd7e074c6"
server_key_version: "encryptionkey@0"
I20260809 08:46:39.891748 29058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:39.905404 29058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:39.907529 29058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:39.907704 29058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:39.912192 29058 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45053
I20260809 08:46:39.912231 29169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45053 every 8 connection(s)
I20260809 08:46:39.912628 29058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:39.913640 29170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:39.914001 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 29058
I20260809 08:46:39.914119 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:39.914309 31997 external_mini_cluster.cc:1428] Setting key b5aba522d49aaec99e29dfd8d546e56f
I20260809 08:46:39.916765 29170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772: Bootstrap starting.
I20260809 08:46:39.917479 29170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:39.917814 29170 log.cc:824] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:39.918586 29170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772: No bootstrap required, opened a new log
I20260809 08:46:39.920930 29170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01810066a66647ad95ab2232c1980772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45053 } }
I20260809 08:46:39.921133 29170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:39.921181 29170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01810066a66647ad95ab2232c1980772, State: Initialized, Role: FOLLOWER
I20260809 08:46:39.921319 29170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01810066a66647ad95ab2232c1980772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45053 } }
I20260809 08:46:39.921406 29170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:39.921442 29170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:39.921495 29170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 08:46:39 dist-test-slave-gkw7 krb5kdc[29042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:46:39.922287 29170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01810066a66647ad95ab2232c1980772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45053 } }
I20260809 08:46:39.922425 29170 leader_election.cc:302] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01810066a66647ad95ab2232c1980772; no voters:
I20260809 08:46:39.922611 29170 leader_election.cc:288] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:39.922888 29170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:39.922652 29177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:39.923099 29177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [term 1 LEADER]: Becoming Leader. State: Replica: 01810066a66647ad95ab2232c1980772, State: Running, Role: LEADER
I20260809 08:46:39.923192 29177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01810066a66647ad95ab2232c1980772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45053 } }
I20260809 08:46:39.923887 29179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01810066a66647ad95ab2232c1980772. Latest consensus state: current_term: 1 leader_uuid: "01810066a66647ad95ab2232c1980772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01810066a66647ad95ab2232c1980772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45053 } } }
I20260809 08:46:39.923978 29179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:39.924048 29178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01810066a66647ad95ab2232c1980772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01810066a66647ad95ab2232c1980772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45053 } } }
I20260809 08:46:39.924117 29178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:39.924197 29190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:39.924671 29190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:39.926488 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:39.914935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33971 (local address 127.0.0.1:45053)
0809 08:46:39.915144 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:39.915154 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:39.915189 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:46:39.915741 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:39.915751 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:39.915812 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:39.915968 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:39.916579 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:39.918016 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:39.918912 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:39.919512 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:39.919635 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:39.922378 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:39.922383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:39.922383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:39.922397 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:39.924665 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:39.925140 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:39.925143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:39.925144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:39.925250 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:39.925509 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:39.925509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:39.925510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:39.925860 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:39.926029 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:39.926178 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 08:46:39.926295 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":85,"threads_started":1}
I20260809 08:46:39.926589 29190 catalog_manager.cc:1347] Generated new cluster ID: 817e3d02a3c94441be8838a1cf038590
I20260809 08:46:39.926627 29190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:39.927635 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:39.936805 29175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:39.929109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48941 (local address 127.0.0.1:45053)
0809 08:46:39.929142 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:39.929144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:39.929267 (+ 123us) server_negotiation.cc:407] Connection header received
0809 08:46:39.929325 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:39.929325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:39.929334 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:39.929373 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:39.930051 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:39.931066 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:39.932099 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:39.932420 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:39.932478 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:39.933952 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:39.933953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:39.933954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:39.933960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:39.935544 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:39.935982 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:39.935983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:39.935984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:39.936040 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:39.936277 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:39.936278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:39.936278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:39.936392 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:39.936481 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:39.936655 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 08:46:39.936716 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:46:39.949726 29190 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:39.950486 29190 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:39.958216 29190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01810066a66647ad95ab2232c1980772: Generated new TSK 0
I20260809 08:46:39.958468 29190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:39.964607 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 29058
W20260809 08:46:39.973994 29040 connection.cc:535] client connection to 127.0.0.1:45053 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45053): Connection reset by peer (error 104)
2026-08-09T08:46:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1534 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:40 dist-test-slave-gkw7 krb5kdc[29219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:40 dist-test-slave-gkw7 krb5kdc[29219](info): set up 2 sockets
Aug 09 08:46:40 dist-test-slave-gkw7 krb5kdc[29219](info): commencing operation
krb5kdc: starting...
W20260809 08:46:41.187644 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:41 dist-test-slave-gkw7 krb5kdc[29219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:41.320456 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:41.404625 29236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:41.404843 29236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:41.404866 29236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:41.406064 29236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:41.406112 29236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:41.406128 29236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:41.406141 29236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:41.406153 29236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:41.407589 29236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36873
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:41.407781 29236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:41.407963 29236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:41.413076 29243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:41.413031 29244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:41.413035 29248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:41.413287 29236 server_base.cc:1034] running on GCE node
I20260809 08:46:41.413514 29236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:41.413888 29236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:41.415038 29236 hybrid_clock.cc:648] HybridClock initialized: now 1786265201415026 us; error 34 us; skew 500 ppm
Aug 09 08:46:41 dist-test-slave-gkw7 krb5kdc[29219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:41.420849 29236 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:41.421355 29236 webserver.cc:460] Webserver started at http://127.0.0.1:35019/ using document root <none> and password file <none>
I20260809 08:46:41.421680 29236 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:41.421756 29236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:41.421936 29236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:41.423040 29236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "b62d78ce35284f9aaa8a2dbbc49957da"
format_stamp: "Formatted at 2026-08-09 08:46:41 on dist-test-slave-gkw7"
server_key: "7ae68f0175015df188b8a9f578b73fc6"
server_key_iv: "107ae37f23e481235cb5a2562af31a1e"
server_key_version: "encryptionkey@0"
I20260809 08:46:41.423424 29236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "b62d78ce35284f9aaa8a2dbbc49957da"
format_stamp: "Formatted at 2026-08-09 08:46:41 on dist-test-slave-gkw7"
server_key: "7ae68f0175015df188b8a9f578b73fc6"
server_key_iv: "107ae37f23e481235cb5a2562af31a1e"
server_key_version: "encryptionkey@0"
I20260809 08:46:41.424813 29236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:46:41.425666 29256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:41.425863 29236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:46:41.425969 29236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "b62d78ce35284f9aaa8a2dbbc49957da"
format_stamp: "Formatted at 2026-08-09 08:46:41 on dist-test-slave-gkw7"
server_key: "7ae68f0175015df188b8a9f578b73fc6"
server_key_iv: "107ae37f23e481235cb5a2562af31a1e"
server_key_version: "encryptionkey@0"
I20260809 08:46:41.426056 29236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:41.446594 29236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:41.449038 29236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:41.449268 29236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:41.453819 29236 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36873
I20260809 08:46:41.453858 29338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36873 every 8 connection(s)
I20260809 08:46:41.454244 29236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:41.455156 29340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:41.456404 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 29236
I20260809 08:46:41.456523 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:41.456743 31997 external_mini_cluster.cc:1428] Setting key 50cca52b5f2b77dba29283df529d15ec
I20260809 08:46:41.457981 29340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da: Bootstrap starting.
I20260809 08:46:41.458726 29340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:41.459026 29340 log.cc:824] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:41.459820 29340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da: No bootstrap required, opened a new log
I20260809 08:46:41.462194 29340 raft_consensus.cc:348] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62d78ce35284f9aaa8a2dbbc49957da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36873 } }
I20260809 08:46:41.462393 29340 raft_consensus.cc:374] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:41.462435 29340 raft_consensus.cc:729] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b62d78ce35284f9aaa8a2dbbc49957da, State: Initialized, Role: FOLLOWER
I20260809 08:46:41.462565 29340 consensus_queue.cc:260] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62d78ce35284f9aaa8a2dbbc49957da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36873 } }
I20260809 08:46:41.462651 29340 raft_consensus.cc:388] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:41.462673 29340 raft_consensus.cc:482] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:41.462710 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:41.457324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43931 (local address 127.0.0.1:36873)
0809 08:46:41.457522 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:41.457570 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:41.457622 (+ 52us) server_negotiation.cc:407] Connection header received
0809 08:46:41.458021 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:41.458049 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:41.458118 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:41.458242 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:41.458521 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:41.459528 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:41.460634 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:41.461044 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:41.461128 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:41.461956 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:41.461960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:41.461960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:41.462130 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:41.462161 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:41.462172 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:41.462238 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:41.462399 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 08:46:41.462439 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260809 08:46:41.462738 29340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:41.463310 29340 raft_consensus.cc:504] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62d78ce35284f9aaa8a2dbbc49957da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36873 } }
I20260809 08:46:41.463426 29340 leader_election.cc:302] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b62d78ce35284f9aaa8a2dbbc49957da; no voters:
I20260809 08:46:41.463630 29340 leader_election.cc:288] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:41.463649 29346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:41.463683 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:41.463819 29346 raft_consensus.cc:686] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [term 1 LEADER]: Becoming Leader. State: Replica: b62d78ce35284f9aaa8a2dbbc49957da, State: Running, Role: LEADER
I20260809 08:46:41.463897 29340 sys_catalog.cc:564] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:41.463903 29346 consensus_queue.cc:237] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62d78ce35284f9aaa8a2dbbc49957da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36873 } }
I20260809 08:46:41.464242 29346 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [sys.catalog]: SysCatalogTable state changed. Reason: New leader b62d78ce35284f9aaa8a2dbbc49957da. Latest consensus state: current_term: 1 leader_uuid: "b62d78ce35284f9aaa8a2dbbc49957da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62d78ce35284f9aaa8a2dbbc49957da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36873 } } }
I20260809 08:46:41.464346 29346 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:41.464253 29352 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b62d78ce35284f9aaa8a2dbbc49957da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62d78ce35284f9aaa8a2dbbc49957da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36873 } } }
I20260809 08:46:41.464509 29352 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:41.464566 29359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:41.466199 29359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:41.467800 29359 catalog_manager.cc:1347] Generated new cluster ID: caabeeb9c75e4fefb32852753d913975
I20260809 08:46:41.467854 29359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:41.473107 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:41.466244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36973 (local address 127.0.0.1:36873)
0809 08:46:41.466283 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:41.466285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:41.466293 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:46:41.469666 (+ 3373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:41.469667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:41.469681 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:41.469750 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:41.469896 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:41.470667 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:41.471659 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:41.471989 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:41.472069 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:41.472834 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:41.472836 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:41.472836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:41.472841 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:41.472859 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:41.472863 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:41.472932 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:41.473005 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 08:46:41.473016 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:46:41.478911 29359 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:41.479511 29359 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:41.483840 29359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b62d78ce35284f9aaa8a2dbbc49957da: Generated new TSK 0
I20260809 08:46:41.484035 29359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:41.492415 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 29236
W20260809 08:46:41.504043 29216 connection.cc:535] client connection to 127.0.0.1:36873 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36873): Connection reset by peer (error 104)
2026-08-09T08:46:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:41 dist-test-slave-gkw7 krb5kdc[29391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:41 dist-test-slave-gkw7 krb5kdc[29391](info): set up 2 sockets
Aug 09 08:46:41 dist-test-slave-gkw7 krb5kdc[29391](info): commencing operation
krb5kdc: starting...
W20260809 08:46:42.735762 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:42 dist-test-slave-gkw7 krb5kdc[29391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:42.862607 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42613
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:42.947504 29417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:42.947734 29417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:42.947790 29417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:42.948959 29417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:42.949009 29417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:42.949023 29417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:42.949036 29417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:42.949049 29417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:42.950515 29417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42613
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:42.950738 29417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:42.950927 29417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:42.956141 29424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:42.956126 29425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:42.956108 29428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:42.956393 29417 server_base.cc:1034] running on GCE node
I20260809 08:46:42.956678 29417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:42.956975 29417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:42.958138 29417 hybrid_clock.cc:648] HybridClock initialized: now 1786265202958129 us; error 54 us; skew 500 ppm
Aug 09 08:46:42 dist-test-slave-gkw7 krb5kdc[29391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265202, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:42.960901 29417 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:42.961450 29417 webserver.cc:460] Webserver started at http://127.0.0.1:41305/ using document root <none> and password file <none>
I20260809 08:46:42.961741 29417 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:42.961844 29417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:42.962013 29417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:42.963191 29417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "1affcd8cca9c4ce3b561398be53c216e"
format_stamp: "Formatted at 2026-08-09 08:46:42 on dist-test-slave-gkw7"
server_key: "9faaf71d8a0377af57156410f8182e49"
server_key_iv: "1ff073f37fa22d149f00c3d1f30d7ee0"
server_key_version: "encryptionkey@0"
I20260809 08:46:42.963598 29417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "1affcd8cca9c4ce3b561398be53c216e"
format_stamp: "Formatted at 2026-08-09 08:46:42 on dist-test-slave-gkw7"
server_key: "9faaf71d8a0377af57156410f8182e49"
server_key_iv: "1ff073f37fa22d149f00c3d1f30d7ee0"
server_key_version: "encryptionkey@0"
I20260809 08:46:42.965196 29417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:42.966403 29436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:42.966617 29417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:42.966733 29417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "1affcd8cca9c4ce3b561398be53c216e"
format_stamp: "Formatted at 2026-08-09 08:46:42 on dist-test-slave-gkw7"
server_key: "9faaf71d8a0377af57156410f8182e49"
server_key_iv: "1ff073f37fa22d149f00c3d1f30d7ee0"
server_key_version: "encryptionkey@0"
I20260809 08:46:42.966835 29417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:42.970652 29417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:42.972782 29417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:42.972959 29417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:42.977582 29417 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42613
I20260809 08:46:42.977661 29519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42613 every 8 connection(s)
I20260809 08:46:42.978035 29417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:42.978452 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 29417
I20260809 08:46:42.978583 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:42.978802 31997 external_mini_cluster.cc:1428] Setting key b580dd37a0295d857d3f4e3ad2320463
I20260809 08:46:42.979867 29520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:42.982386 29520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e: Bootstrap starting.
I20260809 08:46:42.983139 29520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:42.983439 29520 log.cc:824] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:42.984149 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:42.981774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57323 (local address 127.0.0.1:42613)
0809 08:46:42.982004 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:42.982015 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:42.982046 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:46:42.982590 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:42.982599 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:42.982644 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:42.982821 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:42.983683 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:42.983688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:42.983689 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:42.983695 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:42.983742 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:42.983759 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:42.983858 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:42.983932 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 08:46:42.983977 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260809 08:46:42.984364 29520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e: No bootstrap required, opened a new log
I20260809 08:46:42.985249 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:42.986882 29520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affcd8cca9c4ce3b561398be53c216e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42613 } }
I20260809 08:46:42.987082 29520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:42.987131 29520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1affcd8cca9c4ce3b561398be53c216e, State: Initialized, Role: FOLLOWER
I20260809 08:46:42.987274 29520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affcd8cca9c4ce3b561398be53c216e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42613 } }
I20260809 08:46:42.987393 29520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:42.987442 29520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:42.987494 29520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:42.988245 29520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affcd8cca9c4ce3b561398be53c216e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42613 } }
I20260809 08:46:42.988371 29520 leader_election.cc:302] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1affcd8cca9c4ce3b561398be53c216e; no voters:
I20260809 08:46:42.988372 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:42.987076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46751 (local address 127.0.0.1:42613)
0809 08:46:42.987123 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:42.987125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:42.987270 (+ 145us) server_negotiation.cc:407] Connection header received
0809 08:46:42.987349 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:42.987349 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:42.987355 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:42.987454 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:42.988100 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:42.988101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:42.988101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:42.988105 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:42.988119 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:42.988123 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:42.988180 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:42.988251 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 08:46:42.988263 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:46:42.988557 29520 leader_election.cc:288] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:42.988600 29532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:42.988776 29532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [term 1 LEADER]: Becoming Leader. State: Replica: 1affcd8cca9c4ce3b561398be53c216e, State: Running, Role: LEADER
I20260809 08:46:42.988814 29520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:42.988864 29532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affcd8cca9c4ce3b561398be53c216e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42613 } }
I20260809 08:46:42.989248 29534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1affcd8cca9c4ce3b561398be53c216e. Latest consensus state: current_term: 1 leader_uuid: "1affcd8cca9c4ce3b561398be53c216e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affcd8cca9c4ce3b561398be53c216e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42613 } } }
I20260809 08:46:42.989287 29533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1affcd8cca9c4ce3b561398be53c216e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1affcd8cca9c4ce3b561398be53c216e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42613 } } }
I20260809 08:46:42.989356 29534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:42.989413 29533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e [sys.catalog]: This master's current role is: LEADER
W20260809 08:46:42.990799 29556 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:46:42.990846 29556 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:46:42.990914 29543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:42.991065 29543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:42.992463 29543 catalog_manager.cc:1347] Generated new cluster ID: c8d072640f434a44bbb53f2b7202a3d2
I20260809 08:46:42.992514 29543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:42.999722 29543 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:42.999898 29543 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:43.004688 29543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1affcd8cca9c4ce3b561398be53c216e: Generated new TSK 0
I20260809 08:46:43.004877 29543 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:43.008502 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 29417
2026-08-09T08:46:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:43 dist-test-slave-gkw7 krb5kdc[29562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:43 dist-test-slave-gkw7 krb5kdc[29562](info): set up 2 sockets
Aug 09 08:46:43 dist-test-slave-gkw7 krb5kdc[29562](info): commencing operation
krb5kdc: starting...
W20260809 08:46:44.268707 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:44 dist-test-slave-gkw7 krb5kdc[29562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:44.394284 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:44.479375 29583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:44.479589 29583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:44.479614 29583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:44.480773 29583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:44.480816 29583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:44.480830 29583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:44.480844 29583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:44.480855 29583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:44.482380 29583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:44.482645 29583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:44.482924 29583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:44.487943 29591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:44.487978 29592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:44.487957 29598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:44.488205 29583 server_base.cc:1034] running on GCE node
I20260809 08:46:44.488452 29583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:44.488699 29583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:44.489876 29583 hybrid_clock.cc:648] HybridClock initialized: now 1786265204489864 us; error 30 us; skew 500 ppm
Aug 09 08:46:44 dist-test-slave-gkw7 krb5kdc[29562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265204, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:44.492686 29583 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:44.493139 29583 webserver.cc:460] Webserver started at http://127.0.0.1:42101/ using document root <none> and password file <none>
I20260809 08:46:44.493374 29583 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:44.493445 29583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:44.493636 29583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:44.494743 29583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "88248a36bdd144abb03648834aed08df"
format_stamp: "Formatted at 2026-08-09 08:46:44 on dist-test-slave-gkw7"
server_key: "d7f99cefa4c16ecd1d26a97014525060"
server_key_iv: "59fca5fb9005dd28023fc29fda243d31"
server_key_version: "encryptionkey@0"
I20260809 08:46:44.495116 29583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "88248a36bdd144abb03648834aed08df"
format_stamp: "Formatted at 2026-08-09 08:46:44 on dist-test-slave-gkw7"
server_key: "d7f99cefa4c16ecd1d26a97014525060"
server_key_iv: "59fca5fb9005dd28023fc29fda243d31"
server_key_version: "encryptionkey@0"
I20260809 08:46:44.496555 29583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:46:44.497345 29610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:44.497514 29583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:46:44.497668 29583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "88248a36bdd144abb03648834aed08df"
format_stamp: "Formatted at 2026-08-09 08:46:44 on dist-test-slave-gkw7"
server_key: "d7f99cefa4c16ecd1d26a97014525060"
server_key_iv: "59fca5fb9005dd28023fc29fda243d31"
server_key_version: "encryptionkey@0"
I20260809 08:46:44.497757 29583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:44.512432 29583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:44.514572 29583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:44.514750 29583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:44.519110 29583 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32935
I20260809 08:46:44.519148 29687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32935 every 8 connection(s)
I20260809 08:46:44.519517 29583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:44.519775 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 29583
I20260809 08:46:44.519896 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:44.520102 31997 external_mini_cluster.cc:1428] Setting key fdd3b6c58eeb44e7370c835a3e787a4a
I20260809 08:46:44.520633 29690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:44.523980 29690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df: Bootstrap starting.
I20260809 08:46:44.524686 29690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:44.525046 29690 log.cc:824] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:44.525897 29690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df: No bootstrap required, opened a new log
Aug 09 08:46:44 dist-test-slave-gkw7 krb5kdc[29562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:46:44.528012 29690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88248a36bdd144abb03648834aed08df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260809 08:46:44.528191 29690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:44.528235 29690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88248a36bdd144abb03648834aed08df, State: Initialized, Role: FOLLOWER
I20260809 08:46:44.528374 29690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88248a36bdd144abb03648834aed08df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260809 08:46:44.528468 29690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:44.528535 29690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:44.528587 29690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:44.529323 29690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88248a36bdd144abb03648834aed08df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260809 08:46:44.529438 29690 leader_election.cc:302] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88248a36bdd144abb03648834aed08df; no voters:
I20260809 08:46:44.529634 29690 leader_election.cc:288] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:44.529683 29698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:44.529872 29698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [term 1 LEADER]: Becoming Leader. State: Replica: 88248a36bdd144abb03648834aed08df, State: Running, Role: LEADER
I20260809 08:46:44.529922 29690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:44.529958 29698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88248a36bdd144abb03648834aed08df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } }
I20260809 08:46:44.530297 29698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88248a36bdd144abb03648834aed08df. Latest consensus state: current_term: 1 leader_uuid: "88248a36bdd144abb03648834aed08df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88248a36bdd144abb03648834aed08df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } } }
I20260809 08:46:44.530391 29698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:44.530306 29700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88248a36bdd144abb03648834aed08df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88248a36bdd144abb03648834aed08df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32935 } } }
I20260809 08:46:44.530490 29700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:44.530813 29707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:44.531381 29707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:44.532034 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:44.520678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37215 (local address 127.0.0.1:32935)
0809 08:46:44.520876 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:44.520886 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:44.520918 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:46:44.521436 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:44.521446 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:44.521510 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:44.521704 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:44.522345 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:44.523711 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:44.524456 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:44.525087 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:44.525214 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:44.527660 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:44.527665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:44.527665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:44.527679 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:44.529732 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:44.530188 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:44.530191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:44.530191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:44.530292 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:44.530551 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:44.530552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:44.530553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:44.530870 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:44.531511 (+ 641us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:44.531584 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 08:46:44.531819 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":104,"threads_started":1}
I20260809 08:46:44.533077 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:44.534636 29707 catalog_manager.cc:1347] Generated new cluster ID: a119faf86a5a470096a21182d83b503d
I20260809 08:46:44.534691 29707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:44.543624 29707 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:44.544327 29707 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:44.549921 29707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88248a36bdd144abb03648834aed08df: Generated new TSK 0
I20260809 08:46:44.550093 29707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:44.556011 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:44.541942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44101 (local address 127.0.0.1:32935)
0809 08:46:44.541988 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:44.541989 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:44.549875 (+ 7886us) server_negotiation.cc:407] Connection header received
0809 08:46:44.549969 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:44.549982 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:44.549994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:44.550048 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:44.550680 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:44.551164 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:44.551931 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:44.552137 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:44.552214 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:44.553424 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:44.553426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:44.553426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:44.553433 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:44.554936 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:44.555355 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:44.555356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:44.555357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:44.555410 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:44.555610 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:44.555611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:44.555611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:44.555694 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:44.555763 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:44.555894 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 08:46:44.555953 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:46:44.557968 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 29583
2026-08-09T08:46:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:44 dist-test-slave-gkw7 krb5kdc[29744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:44 dist-test-slave-gkw7 krb5kdc[29744](info): set up 2 sockets
Aug 09 08:46:44 dist-test-slave-gkw7 krb5kdc[29744](info): commencing operation
krb5kdc: starting...
W20260809 08:46:45.796720 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:45 dist-test-slave-gkw7 krb5kdc[29744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:45.918524 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:46.001874 29772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:46.002108 29772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:46.002146 29772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:46.003284 29772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:46.003350 29772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:46.003373 29772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:46.003402 29772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:46.003434 29772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:46.004904 29772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:46.005190 29772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:46.005421 29772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:46.010288 29783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:46.010426 29772 server_base.cc:1034] running on GCE node
W20260809 08:46:46.010293 29784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:46.010270 29789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:46.010874 29772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:46.011215 29772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:46.012379 29772 hybrid_clock.cc:648] HybridClock initialized: now 1786265206012377 us; error 39 us; skew 500 ppm
Aug 09 08:46:46 dist-test-slave-gkw7 krb5kdc[29744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:46.019505 29772 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:46.019927 29772 webserver.cc:460] Webserver started at http://127.0.0.1:33935/ using document root <none> and password file <none>
I20260809 08:46:46.020128 29772 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:46.020176 29772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:46.020270 29772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:46.021271 29772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "d561c79beead457fad5d358f87470989"
format_stamp: "Formatted at 2026-08-09 08:46:46 on dist-test-slave-gkw7"
server_key: "92c75986755fbffa304dc28fa1a104e4"
server_key_iv: "a3adb762269b8a683d3d1d71b0a7352b"
server_key_version: "encryptionkey@0"
I20260809 08:46:46.021643 29772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "d561c79beead457fad5d358f87470989"
format_stamp: "Formatted at 2026-08-09 08:46:46 on dist-test-slave-gkw7"
server_key: "92c75986755fbffa304dc28fa1a104e4"
server_key_iv: "a3adb762269b8a683d3d1d71b0a7352b"
server_key_version: "encryptionkey@0"
I20260809 08:46:46.022996 29772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:46.023753 29806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:46.023969 29772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:46.024109 29772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "d561c79beead457fad5d358f87470989"
format_stamp: "Formatted at 2026-08-09 08:46:46 on dist-test-slave-gkw7"
server_key: "92c75986755fbffa304dc28fa1a104e4"
server_key_iv: "a3adb762269b8a683d3d1d71b0a7352b"
server_key_version: "encryptionkey@0"
I20260809 08:46:46.024181 29772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:46.037021 29772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:46.039415 29772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:46.039566 29772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:46.043901 29772 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43999
I20260809 08:46:46.043991 29895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43999 every 8 connection(s)
I20260809 08:46:46.044318 29772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:46.045228 29896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:46.047554 29896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989: Bootstrap starting.
I20260809 08:46:46.048166 29896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:46.048477 29896 log.cc:824] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:46.049268 29896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989: No bootstrap required, opened a new log
I20260809 08:46:46.051878 29896 raft_consensus.cc:348] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d561c79beead457fad5d358f87470989" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43999 } }
I20260809 08:46:46.052067 29896 raft_consensus.cc:374] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:46.052114 29896 raft_consensus.cc:729] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d561c79beead457fad5d358f87470989, State: Initialized, Role: FOLLOWER
I20260809 08:46:46.052335 29896 consensus_queue.cc:260] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d561c79beead457fad5d358f87470989" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43999 } }
I20260809 08:46:46.052440 29896 raft_consensus.cc:388] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:46.052484 29896 raft_consensus.cc:482] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:46.052544 29896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:46.053351 29896 raft_consensus.cc:504] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d561c79beead457fad5d358f87470989" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43999 } }
I20260809 08:46:46.053483 29896 leader_election.cc:302] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d561c79beead457fad5d358f87470989; no voters:
I20260809 08:46:46.053766 29896 leader_election.cc:288] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:46.053782 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 29772
I20260809 08:46:46.053838 29900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:46.053922 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:46.054068 29900 raft_consensus.cc:686] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [term 1 LEADER]: Becoming Leader. State: Replica: d561c79beead457fad5d358f87470989, State: Running, Role: LEADER
I20260809 08:46:46.054098 29896 sys_catalog.cc:564] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:46.054132 31997 external_mini_cluster.cc:1428] Setting key b8ed73ac5f7595d01a67e8a58b8b2ece
I20260809 08:46:46.054153 29900 consensus_queue.cc:237] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d561c79beead457fad5d358f87470989" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43999 } }
I20260809 08:46:46.054471 29900 sys_catalog.cc:455] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d561c79beead457fad5d358f87470989" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d561c79beead457fad5d358f87470989" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43999 } } }
I20260809 08:46:46.054553 29900 sys_catalog.cc:458] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:46.054684 29900 sys_catalog.cc:455] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d561c79beead457fad5d358f87470989. Latest consensus state: current_term: 1 leader_uuid: "d561c79beead457fad5d358f87470989" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d561c79beead457fad5d358f87470989" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43999 } } }
I20260809 08:46:46.054827 29900 sys_catalog.cc:458] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:46.055787 29908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:46.056640 29908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:46.060357 29908 catalog_manager.cc:1347] Generated new cluster ID: e84a9b7df43648eba27a260286cd0620
I20260809 08:46:46.060408 29908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:46.061590 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:46.055246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47157 (local address 127.0.0.1:43999)
0809 08:46:46.055405 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:46.055416 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:46.055450 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:46:46.055567 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:46.055575 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:46.055641 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:46.055811 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:46.056715 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:46.057908 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:46.058972 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:46.059539 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:46.059629 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:46.060498 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:46.060503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:46.060504 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:46.060769 (+ 265us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:46.060820 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:46.060840 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:46.060910 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:46.061146 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 08:46:46.061249 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260809 08:46:46.062769 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:46.067276 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:46.064294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44605 (local address 127.0.0.1:43999)
0809 08:46:46.064331 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:46.064333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:46.064341 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:46:46.064396 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:46.064397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:46.064407 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:46.064462 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:46.064585 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:46.065063 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:46.065993 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:46.066199 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:46.066285 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:46.066965 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:46.066966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:46.066966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:46.066971 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:46.066986 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:46.066992 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:46.067021 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:46.067128 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 08:46:46.067160 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:46:46.084661 29908 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:46.085440 29908 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:46.096619 29908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d561c79beead457fad5d358f87470989: Generated new TSK 0
I20260809 08:46:46.096885 29908 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:46.102378 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 29772
2026-08-09T08:46:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:46 dist-test-slave-gkw7 krb5kdc[29942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:46 dist-test-slave-gkw7 krb5kdc[29942](info): set up 2 sockets
Aug 09 08:46:46 dist-test-slave-gkw7 krb5kdc[29942](info): commencing operation
krb5kdc: starting...
W20260809 08:46:47.339871 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:47 dist-test-slave-gkw7 krb5kdc[29942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:47.468685 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43081
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:47.552271 29965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:47.552471 29965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:47.552495 29965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:47.553748 29965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:47.553790 29965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:47.553805 29965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:47.553817 29965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:47.553829 29965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:47.555354 29965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:47.555603 29965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:47.555832 29965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:47.560849 29975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:47.560863 29977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:47.560881 29974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:47.560879 29965 server_base.cc:1034] running on GCE node
I20260809 08:46:47.561372 29965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:47.561708 29965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:47.562882 29965 hybrid_clock.cc:648] HybridClock initialized: now 1786265207562879 us; error 41 us; skew 500 ppm
Aug 09 08:46:47 dist-test-slave-gkw7 krb5kdc[29942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:47.570436 29965 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:47.570840 29965 webserver.cc:460] Webserver started at http://127.0.0.1:41847/ using document root <none> and password file <none>
I20260809 08:46:47.571022 29965 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:47.571060 29965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:47.571148 29965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:47.572165 29965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "d655bd5d7f3642799f0da00780914df0"
format_stamp: "Formatted at 2026-08-09 08:46:47 on dist-test-slave-gkw7"
server_key: "6078fcdf02257d0e3e0ef5f8b3492db8"
server_key_iv: "a7518e0082729f4236b28f5a0642bee0"
server_key_version: "encryptionkey@0"
I20260809 08:46:47.572474 29965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "d655bd5d7f3642799f0da00780914df0"
format_stamp: "Formatted at 2026-08-09 08:46:47 on dist-test-slave-gkw7"
server_key: "6078fcdf02257d0e3e0ef5f8b3492db8"
server_key_iv: "a7518e0082729f4236b28f5a0642bee0"
server_key_version: "encryptionkey@0"
I20260809 08:46:47.573825 29965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:46:47.574621 29989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:47.574844 29965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:46:47.574939 29965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "d655bd5d7f3642799f0da00780914df0"
format_stamp: "Formatted at 2026-08-09 08:46:47 on dist-test-slave-gkw7"
server_key: "6078fcdf02257d0e3e0ef5f8b3492db8"
server_key_iv: "a7518e0082729f4236b28f5a0642bee0"
server_key_version: "encryptionkey@0"
I20260809 08:46:47.575013 29965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:47.578418 29965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:47.580324 29965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:47.580466 29965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:47.584812 29965 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43081
I20260809 08:46:47.584878 30071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43081 every 8 connection(s)
I20260809 08:46:47.585199 29965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:47.586266 30074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:47.588526 30074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0: Bootstrap starting.
I20260809 08:46:47.589123 30074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:47.589422 30074 log.cc:824] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:47.590250 30074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0: No bootstrap required, opened a new log
I20260809 08:46:47.591995 30074 raft_consensus.cc:348] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d655bd5d7f3642799f0da00780914df0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } }
I20260809 08:46:47.592139 30074 raft_consensus.cc:374] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:47.592163 30074 raft_consensus.cc:729] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d655bd5d7f3642799f0da00780914df0, State: Initialized, Role: FOLLOWER
I20260809 08:46:47.592319 30074 consensus_queue.cc:260] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d655bd5d7f3642799f0da00780914df0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } }
I20260809 08:46:47.592406 30074 raft_consensus.cc:388] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:47.592432 30074 raft_consensus.cc:482] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:47.592478 30074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:47.593034 30074 raft_consensus.cc:504] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d655bd5d7f3642799f0da00780914df0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } }
I20260809 08:46:47.593142 30074 leader_election.cc:302] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d655bd5d7f3642799f0da00780914df0; no voters:
I20260809 08:46:47.593278 30074 leader_election.cc:288] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:47.593328 30078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:47.593621 30074 sys_catalog.cc:564] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:47.593657 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 29965
I20260809 08:46:47.593756 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:47.593954 31997 external_mini_cluster.cc:1428] Setting key 4a52d6f5280f57241424dfd299630792
I20260809 08:46:47.594131 30078 raft_consensus.cc:686] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [term 1 LEADER]: Becoming Leader. State: Replica: d655bd5d7f3642799f0da00780914df0, State: Running, Role: LEADER
I20260809 08:46:47.594324 30078 consensus_queue.cc:237] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d655bd5d7f3642799f0da00780914df0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } }
I20260809 08:46:47.594913 30080 sys_catalog.cc:455] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d655bd5d7f3642799f0da00780914df0. Latest consensus state: current_term: 1 leader_uuid: "d655bd5d7f3642799f0da00780914df0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d655bd5d7f3642799f0da00780914df0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } } }
I20260809 08:46:47.595016 30080 sys_catalog.cc:458] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:47.595081 30079 sys_catalog.cc:455] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d655bd5d7f3642799f0da00780914df0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d655bd5d7f3642799f0da00780914df0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43081 } } }
I20260809 08:46:47.595655 30079 sys_catalog.cc:458] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:47.595256 30104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:47.596346 30104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:47.596697 30095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:47.594719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50209 (local address 127.0.0.1:43081)
0809 08:46:47.594907 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:47.594919 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:47.594962 (+ 43us) server_negotiation.cc:407] Connection header received
0809 08:46:47.595114 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:47.595122 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:47.595160 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:47.595381 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:47.596079 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:47.596083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:47.596084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:47.596088 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:47.596145 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:47.596160 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:47.596241 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:47.596464 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 08:46:47.596507 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260809 08:46:47.597853 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:47.598047 30104 catalog_manager.cc:1347] Generated new cluster ID: 42ad3e51f410467dbbcc7d7372d3c9c0
I20260809 08:46:47.598099 30104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:47.600715 30095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:47.599657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42659 (local address 127.0.0.1:43081)
0809 08:46:47.599699 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:47.599701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:47.599710 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:46:47.599733 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:47.599734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:47.599738 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:47.599800 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:47.600449 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:47.600450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:47.600451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:47.600456 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:47.600470 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:47.600475 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:47.600539 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:47.600609 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 08:46:47.600623 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:46:47.611931 30104 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:47.612098 30104 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:47.617861 30104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d655bd5d7f3642799f0da00780914df0: Generated new TSK 0
I20260809 08:46:47.618055 30104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:47.623082 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 29965
2026-08-09T08:46:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:47 dist-test-slave-gkw7 krb5kdc[30119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:47 dist-test-slave-gkw7 krb5kdc[30119](info): set up 2 sockets
Aug 09 08:46:47 dist-test-slave-gkw7 krb5kdc[30119](info): commencing operation
krb5kdc: starting...
W20260809 08:46:48.871291 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:48 dist-test-slave-gkw7 krb5kdc[30119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:49.000474 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:49.084664 30139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:49.084879 30139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:49.084904 30139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:49.086097 30139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:49.086146 30139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:49.086161 30139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:49.086174 30139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:49.086185 30139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:49.087633 30139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:49.087826 30139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:49.088028 30139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:46:49.093091 30139 server_base.cc:1034] running on GCE node
W20260809 08:46:49.093112 30149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:49.093058 30148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:49.093050 30151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:49.093438 30139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:49.093803 30139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:49.094965 30139 hybrid_clock.cc:648] HybridClock initialized: now 1786265209094938 us; error 41 us; skew 500 ppm
Aug 09 08:46:49 dist-test-slave-gkw7 krb5kdc[30119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265209, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:49.097779 30139 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:49.098292 30139 webserver.cc:460] Webserver started at http://127.0.0.1:42519/ using document root <none> and password file <none>
I20260809 08:46:49.098567 30139 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:49.098639 30139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:49.098803 30139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:49.099917 30139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "f6b9a58a533249bc94f50d9ec21fda03"
format_stamp: "Formatted at 2026-08-09 08:46:49 on dist-test-slave-gkw7"
server_key: "f17596c95d157d3741543ea3ad54fcec"
server_key_iv: "313035af99499c0a6b8cbd06d40f1435"
server_key_version: "encryptionkey@0"
I20260809 08:46:49.100303 30139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "f6b9a58a533249bc94f50d9ec21fda03"
format_stamp: "Formatted at 2026-08-09 08:46:49 on dist-test-slave-gkw7"
server_key: "f17596c95d157d3741543ea3ad54fcec"
server_key_iv: "313035af99499c0a6b8cbd06d40f1435"
server_key_version: "encryptionkey@0"
I20260809 08:46:49.101750 30139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:49.102557 30161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:49.102761 30139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:49.102859 30139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "f6b9a58a533249bc94f50d9ec21fda03"
format_stamp: "Formatted at 2026-08-09 08:46:49 on dist-test-slave-gkw7"
server_key: "f17596c95d157d3741543ea3ad54fcec"
server_key_iv: "313035af99499c0a6b8cbd06d40f1435"
server_key_version: "encryptionkey@0"
I20260809 08:46:49.102914 30139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:49.114758 30139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:49.117295 30139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:49.117424 30139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:49.121582 30139 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43975
I20260809 08:46:49.121699 30253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43975 every 8 connection(s)
I20260809 08:46:49.122115 30139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:49.123160 30256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:49.125818 30256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03: Bootstrap starting.
I20260809 08:46:49.126377 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 30139
I20260809 08:46:49.126503 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:49.126585 30256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:49.126744 31997 external_mini_cluster.cc:1428] Setting key db5fbce3773f571d6b7e1489877ed6c6
I20260809 08:46:49.127036 30256 log.cc:824] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:49.128083 30256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03: No bootstrap required, opened a new log
I20260809 08:46:49.130771 30256 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b9a58a533249bc94f50d9ec21fda03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } }
I20260809 08:46:49.130952 30256 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:49.130993 30256 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b9a58a533249bc94f50d9ec21fda03, State: Initialized, Role: FOLLOWER
I20260809 08:46:49.131222 30256 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b9a58a533249bc94f50d9ec21fda03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } }
I20260809 08:46:49.131345 30256 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:49.131397 30256 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:49.131450 30256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:49.132164 30256 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b9a58a533249bc94f50d9ec21fda03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } }
I20260809 08:46:49.132287 30256 leader_election.cc:302] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6b9a58a533249bc94f50d9ec21fda03; no voters:
I20260809 08:46:49.132470 30256 leader_election.cc:288] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:49.132548 30263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:49.132759 30256 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:49.132743 30263 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [term 1 LEADER]: Becoming Leader. State: Replica: f6b9a58a533249bc94f50d9ec21fda03, State: Running, Role: LEADER
I20260809 08:46:49.133699 30263 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b9a58a533249bc94f50d9ec21fda03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } }
I20260809 08:46:49.134063 30263 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6b9a58a533249bc94f50d9ec21fda03. Latest consensus state: current_term: 1 leader_uuid: "f6b9a58a533249bc94f50d9ec21fda03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b9a58a533249bc94f50d9ec21fda03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } } }
I20260809 08:46:49.134248 30263 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [sys.catalog]: This master's current role is: LEADER
W20260809 08:46:49.134984 30281 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:46:49.135041 30281 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:46:49.135076 30265 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6b9a58a533249bc94f50d9ec21fda03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b9a58a533249bc94f50d9ec21fda03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43975 } } }
I20260809 08:46:49.135147 30265 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:49.135347 30282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 08:46:49 dist-test-slave-gkw7 krb5kdc[30119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:46:49.135561 30282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:49.137316 30282 catalog_manager.cc:1347] Generated new cluster ID: 230605fda9004004b348f95f64d98d9a
I20260809 08:46:49.137378 30282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:49.139745 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:49.127465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:44921 (local address 127.0.0.1:43975)
0809 08:46:49.127637 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:49.127646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:49.127705 (+ 59us) server_negotiation.cc:407] Connection header received
0809 08:46:49.128405 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:49.128415 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:49.128487 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:49.128657 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:49.129296 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:49.130961 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:49.131902 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:49.132641 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:49.133220 (+ 579us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:49.136026 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:49.136030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:49.136031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:49.136043 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:49.138090 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:49.138512 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:49.138516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:49.138516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:49.138612 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:49.138862 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:49.138863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:49.138864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:49.139108 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:49.139304 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:49.139402 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 08:46:49.139507 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260809 08:46:49.140779 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:49.148669 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:49.142321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:59717 (local address 127.0.0.1:43975)
0809 08:46:49.142362 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:49.142364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:49.142472 (+ 108us) server_negotiation.cc:407] Connection header received
0809 08:46:49.142544 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:49.142545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:49.142555 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:49.142611 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:49.143130 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:49.143652 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:49.144374 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:49.144640 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:49.144731 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:49.146018 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:49.146019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:49.146020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:49.146028 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:49.147451 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:49.147823 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:49.147825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:49.147825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:49.147886 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:49.148118 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:49.148120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:49.148120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:49.148236 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:49.148353 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:49.148477 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 08:46:49.148542 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:46:49.177614 30282 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:49.178191 30282 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:49.185835 30282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6b9a58a533249bc94f50d9ec21fda03: Generated new TSK 0
I20260809 08:46:49.186056 30282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:49.196120 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 30139
W20260809 08:46:49.211633 30115 connection.cc:535] client connection to 127.0.0.1:43975 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43975): Connection reset by peer (error 104)
2026-08-09T08:46:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:49 dist-test-slave-gkw7 krb5kdc[30299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:49 dist-test-slave-gkw7 krb5kdc[30299](info): set up 2 sockets
Aug 09 08:46:49 dist-test-slave-gkw7 krb5kdc[30299](info): commencing operation
krb5kdc: starting...
W20260809 08:46:50.417490 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:50 dist-test-slave-gkw7 krb5kdc[30299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:50.595044 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36433
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:50.757671 30315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:50.757984 30315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:50.758036 30315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:50.759420 30315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:50.759509 30315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:50.759538 30315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:50.759567 30315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:50.759617 30315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:50.761723 30315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:50.762122 30315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:50.762442 30315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:50.769307 30325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:50.769263 30327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:50.769261 30330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:50.769762 30315 server_base.cc:1034] running on GCE node
I20260809 08:46:50.770061 30315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:50.770501 30315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:50.771742 30315 hybrid_clock.cc:648] HybridClock initialized: now 1786265210771715 us; error 88 us; skew 500 ppm
Aug 09 08:46:50 dist-test-slave-gkw7 krb5kdc[30299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265210, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:50.778894 30315 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:50.780177 30315 webserver.cc:460] Webserver started at http://127.0.0.1:40315/ using document root <none> and password file <none>
I20260809 08:46:50.780989 30315 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:50.781267 30315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:50.781862 30315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:50.784348 30315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "8c483f1f04bd49c0a1f3158dbad9201b"
format_stamp: "Formatted at 2026-08-09 08:46:50 on dist-test-slave-gkw7"
server_key: "828c12031b87d7665b18e0ae09c6fdd5"
server_key_iv: "d0bbcd3aa7e2d8307d04e32fc26785b5"
server_key_version: "encryptionkey@0"
I20260809 08:46:50.785192 30315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "8c483f1f04bd49c0a1f3158dbad9201b"
format_stamp: "Formatted at 2026-08-09 08:46:50 on dist-test-slave-gkw7"
server_key: "828c12031b87d7665b18e0ae09c6fdd5"
server_key_iv: "d0bbcd3aa7e2d8307d04e32fc26785b5"
server_key_version: "encryptionkey@0"
I20260809 08:46:50.788590 30315 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 08:46:50.790567 30344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:50.791090 30315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:50.791326 30315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "8c483f1f04bd49c0a1f3158dbad9201b"
format_stamp: "Formatted at 2026-08-09 08:46:50 on dist-test-slave-gkw7"
server_key: "828c12031b87d7665b18e0ae09c6fdd5"
server_key_iv: "d0bbcd3aa7e2d8307d04e32fc26785b5"
server_key_version: "encryptionkey@0"
I20260809 08:46:50.791464 30315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:50.847436 30315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:50.854080 30315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:50.854628 30315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:50.870846 30315 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36433
I20260809 08:46:50.870976 30423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36433 every 8 connection(s)
I20260809 08:46:50.872185 30315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:50.875501 30424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:50.880564 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 30315
I20260809 08:46:50.880995 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:50.882333 31997 external_mini_cluster.cc:1428] Setting key a8a6382931adfd4c7132ca8423ecd7ff
I20260809 08:46:50.886925 30424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b: Bootstrap starting.
I20260809 08:46:50.889106 30424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:50.889895 30424 log.cc:824] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:50.891642 30424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b: No bootstrap required, opened a new log
I20260809 08:46:50.897337 30424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c483f1f04bd49c0a1f3158dbad9201b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260809 08:46:50.898106 30424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:50.898288 30424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c483f1f04bd49c0a1f3158dbad9201b, State: Initialized, Role: FOLLOWER
I20260809 08:46:50.898932 30424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c483f1f04bd49c0a1f3158dbad9201b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260809 08:46:50.899158 30424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:50.899325 30424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:50.899677 30424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:50.900012 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:50.884366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:60411 (local address 127.0.0.1:36433)
0809 08:46:50.884919 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:50.884933 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:50.885061 (+ 128us) server_negotiation.cc:407] Connection header received
0809 08:46:50.886800 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:50.886813 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:50.886930 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:50.887271 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:50.888107 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:50.890742 (+ 2635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:50.892576 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:50.894061 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:50.894810 (+ 749us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:50.897059 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:50.897065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:50.897065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:50.897991 (+ 926us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:50.898086 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:50.898182 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:50.898459 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:50.898984 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 08:46:50.899061 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":194,"threads_started":1}
I20260809 08:46:50.901451 30424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c483f1f04bd49c0a1f3158dbad9201b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260809 08:46:50.901824 30424 leader_election.cc:302] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c483f1f04bd49c0a1f3158dbad9201b; no voters:
I20260809 08:46:50.902335 30424 leader_election.cc:288] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:50.902441 30431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:50.902518 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:50.902974 30431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [term 1 LEADER]: Becoming Leader. State: Replica: 8c483f1f04bd49c0a1f3158dbad9201b, State: Running, Role: LEADER
I20260809 08:46:50.903262 30424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:50.903316 30431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c483f1f04bd49c0a1f3158dbad9201b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260809 08:46:50.904413 30431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c483f1f04bd49c0a1f3158dbad9201b. Latest consensus state: current_term: 1 leader_uuid: "8c483f1f04bd49c0a1f3158dbad9201b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c483f1f04bd49c0a1f3158dbad9201b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } } }
I20260809 08:46:50.904641 30431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:50.905261 30433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c483f1f04bd49c0a1f3158dbad9201b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c483f1f04bd49c0a1f3158dbad9201b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } } }
I20260809 08:46:50.905762 30433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:50.906951 30439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:50.909621 30439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:50.914804 30439 catalog_manager.cc:1347] Generated new cluster ID: 04f6ceb3225742b7855e57ff4545b681
I20260809 08:46:50.914987 30439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:50.918192 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:50.907823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:50775 (local address 127.0.0.1:36433)
0809 08:46:50.907917 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:50.907941 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:50.908261 (+ 320us) server_negotiation.cc:407] Connection header received
0809 08:46:50.908361 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:50.908362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:50.908387 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:50.908515 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:50.908902 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:50.911358 (+ 2456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:50.914712 (+ 3354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:50.915409 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:50.915622 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:50.917459 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:50.917461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:50.917461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:50.917467 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:50.917496 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:50.917510 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:50.917687 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:50.917949 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 08:46:50.918001 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:46:50.934387 30439 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:50.935161 30439 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:50.943496 30439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c483f1f04bd49c0a1f3158dbad9201b: Generated new TSK 0
I20260809 08:46:50.943809 30439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:50.948704 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 30315
W20260809 08:46:50.976406 30297 connection.cc:535] client connection to 127.0.0.1:36433 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36433): Connection reset by peer (error 104)
2026-08-09T08:46:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1767 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:51 dist-test-slave-gkw7 krb5kdc[30472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:51 dist-test-slave-gkw7 krb5kdc[30472](info): set up 2 sockets
Aug 09 08:46:51 dist-test-slave-gkw7 krb5kdc[30472](info): commencing operation
krb5kdc: starting...
W20260809 08:46:52.390919 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.397s user 0.001s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:52 dist-test-slave-gkw7 krb5kdc[30472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:52.520576 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46095
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:52.606392 30502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:52.606596 30502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:52.606619 30502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:52.607797 30502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:52.607842 30502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:52.607856 30502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:52.607869 30502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:52.607913 30502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:52.609421 30502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:52.609716 30502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:52.609961 30502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:52.615088 30510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:52.615120 30502 server_base.cc:1034] running on GCE node
W20260809 08:46:52.615139 30513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:52.615072 30511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:52.615504 30502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:52.615866 30502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:52.617041 30502 hybrid_clock.cc:648] HybridClock initialized: now 1786265212617016 us; error 57 us; skew 500 ppm
Aug 09 08:46:52 dist-test-slave-gkw7 krb5kdc[30472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265212, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:52.623265 30502 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:52.623762 30502 webserver.cc:460] Webserver started at http://127.0.0.1:39639/ using document root <none> and password file <none>
I20260809 08:46:52.624017 30502 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:52.624086 30502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:52.624231 30502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:52.625308 30502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "4641e6f1f8f64d9c9146fa91d8cb4854"
format_stamp: "Formatted at 2026-08-09 08:46:52 on dist-test-slave-gkw7"
server_key: "502bbe2eb245d2c73f0d4c199854717d"
server_key_iv: "6fd450fea5aefa2454224bb39ca90c4c"
server_key_version: "encryptionkey@0"
I20260809 08:46:52.625698 30502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "4641e6f1f8f64d9c9146fa91d8cb4854"
format_stamp: "Formatted at 2026-08-09 08:46:52 on dist-test-slave-gkw7"
server_key: "502bbe2eb245d2c73f0d4c199854717d"
server_key_iv: "6fd450fea5aefa2454224bb39ca90c4c"
server_key_version: "encryptionkey@0"
I20260809 08:46:52.627125 30502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:52.627916 30525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:52.628278 30502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:52.628414 30502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "4641e6f1f8f64d9c9146fa91d8cb4854"
format_stamp: "Formatted at 2026-08-09 08:46:52 on dist-test-slave-gkw7"
server_key: "502bbe2eb245d2c73f0d4c199854717d"
server_key_iv: "6fd450fea5aefa2454224bb39ca90c4c"
server_key_version: "encryptionkey@0"
I20260809 08:46:52.628496 30502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:52.632000 30502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:52.634115 30502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:52.634281 30502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:52.638654 30502 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46095
I20260809 08:46:52.638688 30596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46095 every 8 connection(s)
I20260809 08:46:52.639032 30502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:52.640035 30598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:52.642499 30598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854: Bootstrap starting.
I20260809 08:46:52.643107 30598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:52.643399 30598 log.cc:824] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:52.644073 30598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854: No bootstrap required, opened a new log
I20260809 08:46:52.645776 30598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4641e6f1f8f64d9c9146fa91d8cb4854" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46095 } }
I20260809 08:46:52.645936 30598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:52.645982 30598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4641e6f1f8f64d9c9146fa91d8cb4854, State: Initialized, Role: FOLLOWER
I20260809 08:46:52.646114 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 30502
I20260809 08:46:52.646136 30598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4641e6f1f8f64d9c9146fa91d8cb4854" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46095 } }
I20260809 08:46:52.646227 30598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:52.646255 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:52.646270 30598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:52.646325 30598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:52.646456 31997 external_mini_cluster.cc:1428] Setting key 7a019404986ff8ed15276633b27e5b57
I20260809 08:46:52.646967 30598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4641e6f1f8f64d9c9146fa91d8cb4854" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46095 } }
I20260809 08:46:52.647079 30598 leader_election.cc:302] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4641e6f1f8f64d9c9146fa91d8cb4854; no voters:
I20260809 08:46:52.647221 30598 leader_election.cc:288] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:52.647257 30609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:52.647420 30609 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [term 1 LEADER]: Becoming Leader. State: Replica: 4641e6f1f8f64d9c9146fa91d8cb4854, State: Running, Role: LEADER
I20260809 08:46:52.647471 30598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:52.647512 30609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4641e6f1f8f64d9c9146fa91d8cb4854" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46095 } }
I20260809 08:46:52.647882 30610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4641e6f1f8f64d9c9146fa91d8cb4854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4641e6f1f8f64d9c9146fa91d8cb4854" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46095 } } }
I20260809 08:46:52.647971 30610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:52.648118 30611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4641e6f1f8f64d9c9146fa91d8cb4854. Latest consensus state: current_term: 1 leader_uuid: "4641e6f1f8f64d9c9146fa91d8cb4854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4641e6f1f8f64d9c9146fa91d8cb4854" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46095 } } }
I20260809 08:46:52.648205 30611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:52.648329 30619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:52.648882 30619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:52.649499 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:52.646969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:60733 (local address 127.0.0.1:46095)
0809 08:46:52.647243 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:52.647296 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:52.647376 (+ 80us) server_negotiation.cc:407] Connection header received
0809 08:46:52.647647 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:52.647655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:52.647699 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:52.648276 (+ 577us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:52.648826 (+ 550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:52.648829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:52.648830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:52.648833 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:52.648872 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:52.648887 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:52.649231 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:52.649266 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 08:46:52.649312 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":79,"threads_started":1}
I20260809 08:46:52.650465 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:52.653041 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:52.651927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:54719 (local address 127.0.0.1:46095)
0809 08:46:52.651983 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:52.651985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:52.652095 (+ 110us) server_negotiation.cc:407] Connection header received
0809 08:46:52.652159 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:52.652160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:52.652166 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:52.652222 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:52.652811 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:52.652812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:52.652813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:52.652817 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:52.652831 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:52.652835 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:52.652880 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:52.652933 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 08:46:52.652948 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:46:52.654633 30619 catalog_manager.cc:1347] Generated new cluster ID: 0a0fad59251a4e4fa3644addce7c1cc4
I20260809 08:46:52.654688 30619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:52.665580 30619 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:52.665776 30619 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:52.672290 30619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4641e6f1f8f64d9c9146fa91d8cb4854: Generated new TSK 0
I20260809 08:46:52.672495 30619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:52.678370 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 30502
2026-08-09T08:46:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1709 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:52 dist-test-slave-gkw7 krb5kdc[30658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:52 dist-test-slave-gkw7 krb5kdc[30658](info): set up 2 sockets
Aug 09 08:46:52 dist-test-slave-gkw7 krb5kdc[30658](info): commencing operation
krb5kdc: starting...
W20260809 08:46:53.909935 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:54 dist-test-slave-gkw7 krb5kdc[30658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:54.040524 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35431
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:54.125277 30677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:54.125542 30677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:54.125603 30677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:54.126765 30677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:54.126842 30677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:54.126866 30677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:54.126895 30677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:54.126927 30677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:54.128463 30677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35431
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:54.128762 30677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:54.129009 30677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:54.134131 30683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:54.134142 30682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:54.134150 30689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:54.134536 30677 server_base.cc:1034] running on GCE node
I20260809 08:46:54.134786 30677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:54.135040 30677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:54.136230 30677 hybrid_clock.cc:648] HybridClock initialized: now 1786265214136210 us; error 32 us; skew 500 ppm
Aug 09 08:46:54 dist-test-slave-gkw7 krb5kdc[30658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:54.138971 30677 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:54.139459 30677 webserver.cc:460] Webserver started at http://127.0.0.1:33155/ using document root <none> and password file <none>
I20260809 08:46:54.139710 30677 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:54.139783 30677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:54.139935 30677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:54.141023 30677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "cb44b8af54e349bba7a24e32e36c0c99"
format_stamp: "Formatted at 2026-08-09 08:46:54 on dist-test-slave-gkw7"
server_key: "d2195328df5384a8a5f58262e4b3fa16"
server_key_iv: "e6e0e1766b0b767692ebe99b37ca5c42"
server_key_version: "encryptionkey@0"
I20260809 08:46:54.141402 30677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "cb44b8af54e349bba7a24e32e36c0c99"
format_stamp: "Formatted at 2026-08-09 08:46:54 on dist-test-slave-gkw7"
server_key: "d2195328df5384a8a5f58262e4b3fa16"
server_key_iv: "e6e0e1766b0b767692ebe99b37ca5c42"
server_key_version: "encryptionkey@0"
I20260809 08:46:54.142848 30677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:46:54.143668 30698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:54.143893 30677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:46:54.144004 30677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "cb44b8af54e349bba7a24e32e36c0c99"
format_stamp: "Formatted at 2026-08-09 08:46:54 on dist-test-slave-gkw7"
server_key: "d2195328df5384a8a5f58262e4b3fa16"
server_key_iv: "e6e0e1766b0b767692ebe99b37ca5c42"
server_key_version: "encryptionkey@0"
I20260809 08:46:54.144080 30677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:54.156931 30677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:54.159134 30677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:54.159313 30677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:54.163687 30677 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35431
I20260809 08:46:54.163739 30798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35431 every 8 connection(s)
I20260809 08:46:54.164117 30677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:54.165014 30800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:54.166427 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 30677
I20260809 08:46:54.166570 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:54.166774 31997 external_mini_cluster.cc:1428] Setting key f8337902f579ae828fdfa848ce99d03c
I20260809 08:46:54.168032 30800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99: Bootstrap starting.
I20260809 08:46:54.168831 30800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:54.169135 30800 log.cc:824] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:54.169895 30800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99: No bootstrap required, opened a new log
I20260809 08:46:54.172312 30800 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb44b8af54e349bba7a24e32e36c0c99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35431 } }
I20260809 08:46:54.172502 30800 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:54.172544 30800 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb44b8af54e349bba7a24e32e36c0c99, State: Initialized, Role: FOLLOWER
I20260809 08:46:54.172706 30800 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb44b8af54e349bba7a24e32e36c0c99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35431 } }
I20260809 08:46:54.172799 30800 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:54.172838 30800 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:54.172894 30800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:54.173676 30800 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb44b8af54e349bba7a24e32e36c0c99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35431 } }
I20260809 08:46:54.173768 30800 leader_election.cc:302] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb44b8af54e349bba7a24e32e36c0c99; no voters:
I20260809 08:46:54.173986 30800 leader_election.cc:288] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:54.174010 30807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:54.174139 30807 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [term 1 LEADER]: Becoming Leader. State: Replica: cb44b8af54e349bba7a24e32e36c0c99, State: Running, Role: LEADER
I20260809 08:46:54.174223 30807 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb44b8af54e349bba7a24e32e36c0c99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35431 } }
Aug 09 08:46:54 dist-test-slave-gkw7 krb5kdc[30658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:46:54.174284 30800 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:54.174544 30811 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb44b8af54e349bba7a24e32e36c0c99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb44b8af54e349bba7a24e32e36c0c99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35431 } } }
I20260809 08:46:54.174635 30811 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:54.174829 30810 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb44b8af54e349bba7a24e32e36c0c99. Latest consensus state: current_term: 1 leader_uuid: "cb44b8af54e349bba7a24e32e36c0c99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb44b8af54e349bba7a24e32e36c0c99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35431 } } }
I20260809 08:46:54.174908 30810 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:54.175033 30815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:54.175621 30815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:54.178620 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:54.167315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:36619 (local address 127.0.0.1:35431)
0809 08:46:54.167501 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:54.167509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:54.167535 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:46:54.167930 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:54.167936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:54.167982 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:54.168134 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:54.168946 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:54.170153 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:54.171223 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:54.171627 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:54.171705 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:54.174896 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:54.174900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:54.174901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:54.174913 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:54.176964 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:54.177405 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:54.177409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:54.177409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:54.177508 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:54.177786 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:54.177788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:54.177788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:54.178036 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:54.178192 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:54.178340 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 08:46:54.178444 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260809 08:46:54.179720 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:54.181296 30815 catalog_manager.cc:1347] Generated new cluster ID: 2426adf69d154a9bb87bcb2f250ed0b8
I20260809 08:46:54.181348 30815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:54.189504 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:54.181263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:37961 (local address 127.0.0.1:35431)
0809 08:46:54.181299 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:54.181300 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:54.181747 (+ 447us) server_negotiation.cc:407] Connection header received
0809 08:46:54.181779 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:54.181780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:54.181804 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:54.181874 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:54.182464 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:54.183296 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:54.184296 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:54.184667 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:54.184769 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:54.186197 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:54.186198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:54.186199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:54.186208 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:54.188015 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:54.188435 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:54.188436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:54.188436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:54.188512 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:54.188784 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:54.188786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:54.188786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:54.188946 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:54.189081 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:54.189194 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 08:46:54.189271 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:46:54.192625 30815 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:54.193403 30815 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:54.198644 30815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb44b8af54e349bba7a24e32e36c0c99: Generated new TSK 0
I20260809 08:46:54.198865 30815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:54.200789 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 30677
W20260809 08:46:54.219341 30653 connection.cc:535] client connection to 127.0.0.1:35431 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35431): Connection reset by peer (error 104)
2026-08-09T08:46:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1531 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:54 dist-test-slave-gkw7 krb5kdc[30843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:54 dist-test-slave-gkw7 krb5kdc[30843](info): set up 2 sockets
Aug 09 08:46:54 dist-test-slave-gkw7 krb5kdc[30843](info): commencing operation
krb5kdc: starting...
W20260809 08:46:55.426925 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:55 dist-test-slave-gkw7 krb5kdc[30843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:55.556461 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:55.643045 30869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:55.643309 30869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:55.643350 30869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:55.644515 30869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:55.644587 30869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:55.644613 30869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:55.644634 30869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:55.644666 30869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:55.646202 30869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38961
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:55.646481 30869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:55.646724 30869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:55.652091 30876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:55.652083 30879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:55.652146 30875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:55.652347 30869 server_base.cc:1034] running on GCE node
I20260809 08:46:55.652717 30869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:55.653079 30869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:55.654228 30869 hybrid_clock.cc:648] HybridClock initialized: now 1786265215654204 us; error 49 us; skew 500 ppm
Aug 09 08:46:55 dist-test-slave-gkw7 krb5kdc[30843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265215, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:55.656867 30869 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:55.657352 30869 webserver.cc:460] Webserver started at http://127.0.0.1:46117/ using document root <none> and password file <none>
I20260809 08:46:55.657677 30869 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:55.657750 30869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:55.657915 30869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:55.659024 30869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "fd529da8e4764927a13b383a3fea8b7a"
format_stamp: "Formatted at 2026-08-09 08:46:55 on dist-test-slave-gkw7"
server_key: "a19c3bdf084b5bd2006797ab83ccaf65"
server_key_iv: "fe522ccd9a07930b410f4163803f3e1c"
server_key_version: "encryptionkey@0"
I20260809 08:46:55.659421 30869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "fd529da8e4764927a13b383a3fea8b7a"
format_stamp: "Formatted at 2026-08-09 08:46:55 on dist-test-slave-gkw7"
server_key: "a19c3bdf084b5bd2006797ab83ccaf65"
server_key_iv: "fe522ccd9a07930b410f4163803f3e1c"
server_key_version: "encryptionkey@0"
I20260809 08:46:55.660852 30869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:46:55.661708 30887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:55.662072 30869 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:55.662207 30869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "fd529da8e4764927a13b383a3fea8b7a"
format_stamp: "Formatted at 2026-08-09 08:46:55 on dist-test-slave-gkw7"
server_key: "a19c3bdf084b5bd2006797ab83ccaf65"
server_key_iv: "fe522ccd9a07930b410f4163803f3e1c"
server_key_version: "encryptionkey@0"
I20260809 08:46:55.662312 30869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:55.674031 30869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:55.676900 30869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:55.677073 30869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:55.681509 30869 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38961
I20260809 08:46:55.681620 30969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38961 every 8 connection(s)
I20260809 08:46:55.681995 30869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:55.682504 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 30869
I20260809 08:46:55.682636 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:55.682834 31997 external_mini_cluster.cc:1428] Setting key 8bb611f5226171f82a4dbd81a9e6854f
I20260809 08:46:55.683151 30970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:55.686791 30970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a: Bootstrap starting.
I20260809 08:46:55.687517 30970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:55.687860 30970 log.cc:824] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:55.688653 30970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a: No bootstrap required, opened a new log
I20260809 08:46:55.689342 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:55.683460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:52923 (local address 127.0.0.1:38961)
0809 08:46:55.683681 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:55.683691 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:55.683725 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:46:55.684470 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:55.684479 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:55.684547 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:55.684706 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:55.684891 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:55.686304 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:55.687118 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:55.687751 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:55.687871 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:55.688597 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:55.688602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:55.688602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:55.688830 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:55.688870 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:55.688885 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:55.688968 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:55.689080 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 08:46:55.689111 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260809 08:46:55.690510 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:55.691084 30970 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd529da8e4764927a13b383a3fea8b7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38961 } }
I20260809 08:46:55.691301 30970 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:55.691351 30970 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd529da8e4764927a13b383a3fea8b7a, State: Initialized, Role: FOLLOWER
I20260809 08:46:55.691468 30970 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd529da8e4764927a13b383a3fea8b7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38961 } }
I20260809 08:46:55.691548 30970 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:55.691596 30970 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:55.691655 30970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:55.692447 30970 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd529da8e4764927a13b383a3fea8b7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38961 } }
I20260809 08:46:55.692566 30970 leader_election.cc:302] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd529da8e4764927a13b383a3fea8b7a; no voters:
I20260809 08:46:55.692759 30970 leader_election.cc:288] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:55.692803 30987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:55.692927 30987 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [term 1 LEADER]: Becoming Leader. State: Replica: fd529da8e4764927a13b383a3fea8b7a, State: Running, Role: LEADER
I20260809 08:46:55.693023 30970 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:55.693019 30987 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd529da8e4764927a13b383a3fea8b7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38961 } }
I20260809 08:46:55.693400 30987 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd529da8e4764927a13b383a3fea8b7a. Latest consensus state: current_term: 1 leader_uuid: "fd529da8e4764927a13b383a3fea8b7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd529da8e4764927a13b383a3fea8b7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38961 } } }
I20260809 08:46:55.693476 30987 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:55.693662 30990 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd529da8e4764927a13b383a3fea8b7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd529da8e4764927a13b383a3fea8b7a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38961 } } }
I20260809 08:46:55.693717 30994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:55.693744 30990 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:55.694380 30994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:55.695932 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:55.692175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:43101 (local address 127.0.0.1:38961)
0809 08:46:55.692236 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:55.692238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:55.692364 (+ 126us) server_negotiation.cc:407] Connection header received
0809 08:46:55.692424 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:55.692424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:55.692432 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:55.692480 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:55.692613 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:55.693295 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:55.694353 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:55.694646 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:55.694710 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:55.695602 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:55.695604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:55.695604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:55.695608 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:55.695622 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:55.695626 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:55.695684 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:55.695852 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 08:46:55.695862 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:46:55.696439 30994 catalog_manager.cc:1347] Generated new cluster ID: 2bff1bff7fd94979bb127a8ba1de55e0
I20260809 08:46:55.696486 30994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:55.721161 30994 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:55.721860 30994 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:55.727973 30994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd529da8e4764927a13b383a3fea8b7a: Generated new TSK 0
I20260809 08:46:55.728173 30994 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:55.736362 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 30869
W20260809 08:46:55.752087 30840 connection.cc:535] client connection to 127.0.0.1:38961 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38961): Connection reset by peer (error 104)
2026-08-09T08:46:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:55 dist-test-slave-gkw7 krb5kdc[31026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:55 dist-test-slave-gkw7 krb5kdc[31026](info): set up 2 sockets
Aug 09 08:46:55 dist-test-slave-gkw7 krb5kdc[31026](info): commencing operation
krb5kdc: starting...
W20260809 08:46:56.996722 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:57 dist-test-slave-gkw7 krb5kdc[31026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:46:57.124851 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44157
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:57.209110 31048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:57.209367 31048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:57.209406 31048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:57.210580 31048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:57.210654 31048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:57.210677 31048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:57.210700 31048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:57.210728 31048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:57.212240 31048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44157
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:57.212523 31048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:57.212759 31048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:57.217903 31058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:57.217995 31060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:57.218187 31048 server_base.cc:1034] running on GCE node
W20260809 08:46:57.217904 31062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:57.218482 31048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:57.218791 31048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:57.219995 31048 hybrid_clock.cc:648] HybridClock initialized: now 1786265217219977 us; error 73 us; skew 500 ppm
Aug 09 08:46:57 dist-test-slave-gkw7 krb5kdc[31026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265217, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:57.222631 31048 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:46:57.223110 31048 webserver.cc:460] Webserver started at http://127.0.0.1:42745/ using document root <none> and password file <none>
I20260809 08:46:57.223320 31048 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:57.223361 31048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:57.223451 31048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:57.224479 31048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d"
format_stamp: "Formatted at 2026-08-09 08:46:57 on dist-test-slave-gkw7"
server_key: "d7b334ab4cf38a0dc007d8f3c7ea4abf"
server_key_iv: "50f1bb175078ac1a7f9098aec9cd61d8"
server_key_version: "encryptionkey@0"
I20260809 08:46:57.224795 31048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d"
format_stamp: "Formatted at 2026-08-09 08:46:57 on dist-test-slave-gkw7"
server_key: "d7b334ab4cf38a0dc007d8f3c7ea4abf"
server_key_iv: "50f1bb175078ac1a7f9098aec9cd61d8"
server_key_version: "encryptionkey@0"
I20260809 08:46:57.226215 31048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:46:57.226914 31071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:57.227103 31048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:46:57.227193 31048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d"
format_stamp: "Formatted at 2026-08-09 08:46:57 on dist-test-slave-gkw7"
server_key: "d7b334ab4cf38a0dc007d8f3c7ea4abf"
server_key_iv: "50f1bb175078ac1a7f9098aec9cd61d8"
server_key_version: "encryptionkey@0"
I20260809 08:46:57.227248 31048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:57.230615 31048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:57.232468 31048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:57.232575 31048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:57.236447 31048 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44157
I20260809 08:46:57.236531 31153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44157 every 8 connection(s)
I20260809 08:46:57.236860 31048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:57.238098 31154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:57.240362 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 31048
I20260809 08:46:57.240391 31154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d: Bootstrap starting.
I20260809 08:46:57.240507 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:57.240716 31997 external_mini_cluster.cc:1428] Setting key fd991e8166d9a027ea2df2d9edc06095
I20260809 08:46:57.241086 31154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:57.241447 31154 log.cc:824] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:57.242220 31154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d: No bootstrap required, opened a new log
I20260809 08:46:57.243435 31160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:57.241215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:45303 (local address 127.0.0.1:44157)
0809 08:46:57.241452 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:57.241460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:57.241485 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:46:57.242012 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:57.242019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:57.242044 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:57.242203 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:57.242969 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:57.242972 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:57.242973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:57.242978 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:57.243024 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:57.243035 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:57.243128 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:57.243221 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 08:46:57.243266 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
I20260809 08:46:57.244477 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:57.244695 31154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44157 } }
I20260809 08:46:57.244865 31154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:57.244918 31154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7876d7ff64bb4ea6b6f4bb50fd516d9d, State: Initialized, Role: FOLLOWER
I20260809 08:46:57.245034 31154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44157 } }
I20260809 08:46:57.245126 31154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:57.245154 31154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:57.245224 31154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:57.245940 31154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44157 } }
I20260809 08:46:57.246119 31154 leader_election.cc:302] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7876d7ff64bb4ea6b6f4bb50fd516d9d; no voters:
I20260809 08:46:57.246315 31154 leader_election.cc:288] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:57.246335 31167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:57.246454 31167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [term 1 LEADER]: Becoming Leader. State: Replica: 7876d7ff64bb4ea6b6f4bb50fd516d9d, State: Running, Role: LEADER
I20260809 08:46:57.246543 31167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44157 } }
I20260809 08:46:57.246613 31154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:57.246892 31167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7876d7ff64bb4ea6b6f4bb50fd516d9d. Latest consensus state: current_term: 1 leader_uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44157 } } }
I20260809 08:46:57.247022 31167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:57.247140 31160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:57.245986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:39735 (local address 127.0.0.1:44157)
0809 08:46:57.246062 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:57.246063 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:57.246127 (+ 64us) server_negotiation.cc:407] Connection header received
0809 08:46:57.246214 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:57.246215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:57.246221 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:57.246274 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:57.246886 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:57.246887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:57.246887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:46:57.246892 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:57.246907 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:46:57.246911 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:57.246943 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:57.247048 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 08:46:57.247061 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 08:46:57.246902 31169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7876d7ff64bb4ea6b6f4bb50fd516d9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44157 } } }
I20260809 08:46:57.247301 31169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:57.247548 31175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:57.248152 31175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:57.249908 31175 catalog_manager.cc:1347] Generated new cluster ID: 89b3de24fed940faa9c86e6814efa44b
I20260809 08:46:57.249967 31175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:46:57.276940 31175 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:57.277215 31175 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:57.282675 31175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7876d7ff64bb4ea6b6f4bb50fd516d9d: Generated new TSK 0
I20260809 08:46:57.282876 31175 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:57.291870 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 31048
2026-08-09T08:46:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1552 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:57 dist-test-slave-gkw7 krb5kdc[31208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:57 dist-test-slave-gkw7 krb5kdc[31208](info): set up 2 sockets
Aug 09 08:46:57 dist-test-slave-gkw7 krb5kdc[31208](info): commencing operation
krb5kdc: starting...
W20260809 08:46:58.524745 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:46:58 dist-test-slave-gkw7 krb5kdc[31208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:46:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:46:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:46:58.651861 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:46103
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:46103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:46:58.734040 31233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:46:58.734280 31233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:46:58.734318 31233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:46:58.735440 31233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:46:58.735509 31233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:46:58.735534 31233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:46:58.735563 31233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:46:58.735595 31233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:46:58.737056 31233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:46103
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:46103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:46:58.737347 31233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:46:58.737599 31233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:46:58.742389 31245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:58.742434 31241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:46:58.742390 31240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:46:58.742733 31233 server_base.cc:1034] running on GCE node
I20260809 08:46:58.742967 31233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:46:58.743259 31233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:46:58.744431 31233 hybrid_clock.cc:648] HybridClock initialized: now 1786265218744413 us; error 34 us; skew 500 ppm
Aug 09 08:46:58 dist-test-slave-gkw7 krb5kdc[31208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265218, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:46:58.752063 31233 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:46:58.752566 31233 webserver.cc:460] Webserver started at http://127.31.63.126:43271/ using document root <none> and password file <none>
I20260809 08:46:58.752816 31233 fs_manager.cc:359] Metadata directory not provided
I20260809 08:46:58.752863 31233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:46:58.752997 31233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:46:58.754114 31233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "abb1d83429934cc981bb7a830a7643b7"
format_stamp: "Formatted at 2026-08-09 08:46:58 on dist-test-slave-gkw7"
server_key: "6695f137300bee23a5f6dc9ee740c379"
server_key_iv: "6d326226a232218457d5c931712ecca4"
server_key_version: "encryptionkey@0"
I20260809 08:46:58.754475 31233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "abb1d83429934cc981bb7a830a7643b7"
format_stamp: "Formatted at 2026-08-09 08:46:58 on dist-test-slave-gkw7"
server_key: "6695f137300bee23a5f6dc9ee740c379"
server_key_iv: "6d326226a232218457d5c931712ecca4"
server_key_version: "encryptionkey@0"
I20260809 08:46:58.755800 31233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:46:58.756572 31254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:46:58.756781 31233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:46:58.756889 31233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "abb1d83429934cc981bb7a830a7643b7"
format_stamp: "Formatted at 2026-08-09 08:46:58 on dist-test-slave-gkw7"
server_key: "6695f137300bee23a5f6dc9ee740c379"
server_key_iv: "6d326226a232218457d5c931712ecca4"
server_key_version: "encryptionkey@0"
I20260809 08:46:58.756964 31233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:46:58.771574 31233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:46:58.773376 31233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:46:58.773579 31233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:46:58.777850 31233 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:46103
I20260809 08:46:58.777884 31325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:46103 every 8 connection(s)
I20260809 08:46:58.778244 31233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:46:58.779141 31326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:46:58.781347 31326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7: Bootstrap starting.
I20260809 08:46:58.781919 31326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7: Neither blocks nor log segments found. Creating new log.
I20260809 08:46:58.782172 31326 log.cc:824] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:46:58.782861 31326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7: No bootstrap required, opened a new log
I20260809 08:46:58.784384 31326 raft_consensus.cc:348] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb1d83429934cc981bb7a830a7643b7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46103 } }
I20260809 08:46:58.784520 31326 raft_consensus.cc:374] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:46:58.784543 31326 raft_consensus.cc:729] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb1d83429934cc981bb7a830a7643b7, State: Initialized, Role: FOLLOWER
I20260809 08:46:58.784700 31326 consensus_queue.cc:260] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb1d83429934cc981bb7a830a7643b7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46103 } }
I20260809 08:46:58.784793 31326 raft_consensus.cc:388] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:46:58.784818 31326 raft_consensus.cc:482] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:46:58.784852 31326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:46:58.785760 31326 raft_consensus.cc:504] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb1d83429934cc981bb7a830a7643b7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46103 } }
I20260809 08:46:58.785846 31326 leader_election.cc:302] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abb1d83429934cc981bb7a830a7643b7; no voters:
I20260809 08:46:58.785975 31326 leader_election.cc:288] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:46:58.786046 31329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:46:58.786208 31329 raft_consensus.cc:686] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [term 1 LEADER]: Becoming Leader. State: Replica: abb1d83429934cc981bb7a830a7643b7, State: Running, Role: LEADER
I20260809 08:46:58.786264 31326 sys_catalog.cc:564] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:46:58.786363 31329 consensus_queue.cc:237] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb1d83429934cc981bb7a830a7643b7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46103 } }
I20260809 08:46:58.786755 31333 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abb1d83429934cc981bb7a830a7643b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb1d83429934cc981bb7a830a7643b7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46103 } } }
I20260809 08:46:58.786859 31333 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:58.786885 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 31233
I20260809 08:46:58.787009 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:46:58.787045 31335 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abb1d83429934cc981bb7a830a7643b7. Latest consensus state: current_term: 1 leader_uuid: "abb1d83429934cc981bb7a830a7643b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb1d83429934cc981bb7a830a7643b7" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 46103 } } }
I20260809 08:46:58.787123 31335 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7 [sys.catalog]: This master's current role is: LEADER
I20260809 08:46:58.787226 31997 external_mini_cluster.cc:1428] Setting key 4cbfdb1d1a21c4098fdcf6b4cd6ae953
I20260809 08:46:58.787253 31344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:46:58.787788 31344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:46:58.789474 31344 catalog_manager.cc:1347] Generated new cluster ID: 1e9e9d23ec0e42dbb3c708c3c1360ef2
I20260809 08:46:58.789546 31344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:46:58 dist-test-slave-gkw7 krb5kdc[31208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:46:58.798372 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:58.787889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54683 (local address 127.31.63.126:46103)
0809 08:46:58.788082 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:58.788092 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:58.788124 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:46:58.788243 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:58.788255 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:58.788297 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:58.788433 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:58.789165 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:58.790224 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:58.791005 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:58.791440 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:58.791580 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:58.794356 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:58.794360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:58.794360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:58.794372 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:58.796585 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:58.796979 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:58.796982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:58.796983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:58.797085 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:58.797328 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:58.797330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:58.797330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:58.797629 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:58.797786 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:58.798039 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 08:46:58.798144 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260809 08:46:58.799418 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:46:58.807097 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:46:58.800834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44367 (local address 127.31.63.126:46103)
0809 08:46:58.800871 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:46:58.800873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:46:58.800997 (+ 124us) server_negotiation.cc:407] Connection header received
0809 08:46:58.801069 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:46:58.801070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:46:58.801080 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:46:58.801123 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:46:58.801681 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:58.802207 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:58.802924 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:46:58.803144 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:46:58.803232 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:46:58.804387 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:46:58.804389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:46:58.804389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:46:58.804396 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:46:58.805870 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:58.806250 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:58.806252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:58.806252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:58.806308 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:46:58.806569 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:46:58.806570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:46:58.806570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:46:58.806679 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:46:58.806790 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 08:46:58.806914 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 08:46:58.806978 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":19}
I20260809 08:46:58.811864 31344 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:46:58.812430 31344 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:46:58.815982 31344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abb1d83429934cc981bb7a830a7643b7: Generated new TSK 0
I20260809 08:46:58.816179 31344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:46:58.821709 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 31233
W20260809 08:46:58.832850 31199 connection.cc:535] client connection to 127.31.63.126:46103 recv error: Network error: failed to read from TLS socket (remote: 127.31.63.126:46103): Connection reset by peer (error 104)
2026-08-09T08:46:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:46:58 dist-test-slave-gkw7 krb5kdc[31368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:46:58 dist-test-slave-gkw7 krb5kdc[31368](info): set up 2 sockets
Aug 09 08:46:58 dist-test-slave-gkw7 krb5kdc[31368](info): commencing operation
krb5kdc: starting...
W20260809 08:47:00.049769 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:47:00 dist-test-slave-gkw7 krb5kdc[31368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:47:00.176767 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:45435
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:45435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:00.260457 31389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:00.260720 31389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:00.260758 31389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:00.262003 31389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:00.262069 31389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:00.262092 31389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:00.262135 31389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:00.262159 31389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:00.263657 31389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:45435
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:45435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:00.263947 31389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:00.264186 31389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:00.269213 31402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:00.269280 31389 server_base.cc:1034] running on GCE node
W20260809 08:47:00.269243 31395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:00.269209 31398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:00.269714 31389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:00.270030 31389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:00.271239 31389 hybrid_clock.cc:648] HybridClock initialized: now 1786265220271211 us; error 37 us; skew 500 ppm
Aug 09 08:47:00 dist-test-slave-gkw7 krb5kdc[31368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265220, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:00.274300 31389 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:47:00.274827 31389 webserver.cc:460] Webserver started at http://127.31.63.126:40717/ using document root <none> and password file <none>
I20260809 08:47:00.275060 31389 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:00.275108 31389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:00.275216 31389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:00.276324 31389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "1fc150c0131045cd893ea82e0dfba56f"
format_stamp: "Formatted at 2026-08-09 08:47:00 on dist-test-slave-gkw7"
server_key: "3210a0f99c4a4f736056aa72837fa933"
server_key_iv: "be571a51df8e8fe212a247491bad8ba4"
server_key_version: "encryptionkey@0"
I20260809 08:47:00.276655 31389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "1fc150c0131045cd893ea82e0dfba56f"
format_stamp: "Formatted at 2026-08-09 08:47:00 on dist-test-slave-gkw7"
server_key: "3210a0f99c4a4f736056aa72837fa933"
server_key_iv: "be571a51df8e8fe212a247491bad8ba4"
server_key_version: "encryptionkey@0"
I20260809 08:47:00.278190 31389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:00.278965 31413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:00.279222 31389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:00.279310 31389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "1fc150c0131045cd893ea82e0dfba56f"
format_stamp: "Formatted at 2026-08-09 08:47:00 on dist-test-slave-gkw7"
server_key: "3210a0f99c4a4f736056aa72837fa933"
server_key_iv: "be571a51df8e8fe212a247491bad8ba4"
server_key_version: "encryptionkey@0"
I20260809 08:47:00.279361 31389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:00.288335 31389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:00.290421 31389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:00.290535 31389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:00.294612 31389 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:45435
I20260809 08:47:00.294682 31486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:45435 every 8 connection(s)
I20260809 08:47:00.295010 31389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:00.296085 31487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:00.298461 31487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f: Bootstrap starting.
I20260809 08:47:00.299067 31487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:00.299360 31487 log.cc:824] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:00.300282 31487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f: No bootstrap required, opened a new log
I20260809 08:47:00.301942 31487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc150c0131045cd893ea82e0dfba56f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45435 } }
I20260809 08:47:00.302124 31487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:00.302166 31487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc150c0131045cd893ea82e0dfba56f, State: Initialized, Role: FOLLOWER
I20260809 08:47:00.302325 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 31389
I20260809 08:47:00.302338 31487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc150c0131045cd893ea82e0dfba56f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45435 } }
I20260809 08:47:00.302423 31487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:00.302470 31487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:00.302492 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:47:00.302525 31487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:00.302708 31997 external_mini_cluster.cc:1428] Setting key 183a8ad3b66065594a7c8058a9558319
I20260809 08:47:00.303743 31487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc150c0131045cd893ea82e0dfba56f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45435 } }
I20260809 08:47:00.303861 31487 leader_election.cc:302] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fc150c0131045cd893ea82e0dfba56f; no voters:
I20260809 08:47:00.304018 31487 leader_election.cc:288] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:00.304040 31495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:00.304231 31495 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [term 1 LEADER]: Becoming Leader. State: Replica: 1fc150c0131045cd893ea82e0dfba56f, State: Running, Role: LEADER
I20260809 08:47:00.304361 31487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:00.304400 31495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc150c0131045cd893ea82e0dfba56f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45435 } }
I20260809 08:47:00.304899 31496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fc150c0131045cd893ea82e0dfba56f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc150c0131045cd893ea82e0dfba56f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45435 } } }
I20260809 08:47:00.304993 31496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:00.305467 31498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fc150c0131045cd893ea82e0dfba56f. Latest consensus state: current_term: 1 leader_uuid: "1fc150c0131045cd893ea82e0dfba56f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fc150c0131045cd893ea82e0dfba56f" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 45435 } } }
I20260809 08:47:00.305646 31498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:00.305565 31507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:00.306239 31507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:00.308741 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:00.303242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50351 (local address 127.31.63.126:45435)
0809 08:47:00.303471 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:00.303481 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:00.303516 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:47:00.303635 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:00.303643 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:00.303701 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:00.303870 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:00.304113 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:00.305476 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:00.306614 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:00.307037 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:00.307189 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:00.308006 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:00.308010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:00.308011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:00.308220 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:00.308253 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:00.308263 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:00.308331 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:00.308489 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 08:47:00.308526 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":116,"threads_started":1}
I20260809 08:47:00.309760 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:00.309950 31507 catalog_manager.cc:1347] Generated new cluster ID: af859b8365a8455db7b587f098550b37
I20260809 08:47:00.309998 31507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:00.319536 31507 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:00.320222 31507 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:00.321751 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:00.317875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48349 (local address 127.31.63.126:45435)
0809 08:47:00.317943 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:00.317945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:00.318061 (+ 116us) server_negotiation.cc:407] Connection header received
0809 08:47:00.318113 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:00.318114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:00.318125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:00.318189 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:00.318321 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:00.319123 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:00.320269 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:00.320603 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:00.320678 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:00.321451 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:00.321453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:00.321453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:00.321458 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:00.321475 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:00.321479 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:00.321599 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:00.321661 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 08:47:00.321674 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:47:00.323987 31507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fc150c0131045cd893ea82e0dfba56f: Generated new TSK 0
I20260809 08:47:00.324178 31507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:00.326678 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 31389
W20260809 08:47:00.342757 31365 connection.cc:535] client connection to 127.31.63.126:45435 recv error: Network error: failed to read from TLS socket (remote: 127.31.63.126:45435): Connection reset by peer (error 104)
2026-08-09T08:47:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:00 dist-test-slave-gkw7 krb5kdc[31531](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:00 dist-test-slave-gkw7 krb5kdc[31531](info): set up 2 sockets
Aug 09 08:47:00 dist-test-slave-gkw7 krb5kdc[31531](info): commencing operation
krb5kdc: starting...
W20260809 08:47:01.570727 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:47:01 dist-test-slave-gkw7 krb5kdc[31531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:47:01.696221 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:44355
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:44097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:44355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:01.781188 31552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:01.781452 31552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:01.781493 31552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:01.782704 31552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:01.782789 31552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:01.782819 31552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:01.782853 31552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:01.782884 31552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:01.784379 31552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:44097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:44355
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:44355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:01.784685 31552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:01.784941 31552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:01.790071 31568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:01.790143 31563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:01.790273 31552 server_base.cc:1034] running on GCE node
W20260809 08:47:01.790141 31562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:01.790632 31552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:01.790982 31552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:01.792156 31552 hybrid_clock.cc:648] HybridClock initialized: now 1786265221792133 us; error 39 us; skew 500 ppm
Aug 09 08:47:01 dist-test-slave-gkw7 krb5kdc[31531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265221, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:01.794775 31552 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:47:01.795274 31552 webserver.cc:460] Webserver started at http://127.31.63.126:36205/ using document root <none> and password file <none>
I20260809 08:47:01.795511 31552 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:01.795590 31552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:01.795756 31552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:01.796850 31552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "7514421488c144d5a9d6ed0aa823b6e2"
format_stamp: "Formatted at 2026-08-09 08:47:01 on dist-test-slave-gkw7"
server_key: "607f9c6844a2c8ce640fe75f9af06f9c"
server_key_iv: "a9e872748a86bfa1e184559b240f8a4d"
server_key_version: "encryptionkey@0"
I20260809 08:47:01.797243 31552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "7514421488c144d5a9d6ed0aa823b6e2"
format_stamp: "Formatted at 2026-08-09 08:47:01 on dist-test-slave-gkw7"
server_key: "607f9c6844a2c8ce640fe75f9af06f9c"
server_key_iv: "a9e872748a86bfa1e184559b240f8a4d"
server_key_version: "encryptionkey@0"
I20260809 08:47:01.798760 31552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:01.799651 31583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:01.799826 31552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:01.799935 31552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "7514421488c144d5a9d6ed0aa823b6e2"
format_stamp: "Formatted at 2026-08-09 08:47:01 on dist-test-slave-gkw7"
server_key: "607f9c6844a2c8ce640fe75f9af06f9c"
server_key_iv: "a9e872748a86bfa1e184559b240f8a4d"
server_key_version: "encryptionkey@0"
I20260809 08:47:01.800029 31552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:01.803519 31552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:01.805697 31552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:01.805872 31552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:01.810295 31552 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:44355
I20260809 08:47:01.810361 31654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:44355 every 8 connection(s)
I20260809 08:47:01.810755 31552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:01.811756 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 31552
I20260809 08:47:01.811770 31656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:01.811908 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:47:01.812103 31997 external_mini_cluster.cc:1428] Setting key 4a55b6426e88e2e44e25cd75b0da45b6
I20260809 08:47:01.814815 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:01.812700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44641 (local address 127.31.63.126:44355)
0809 08:47:01.812922 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:01.812932 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:01.812965 (+ 33us) server_negotiation.cc:407] Connection header received
0809 08:47:01.813469 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:01.813478 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:01.813513 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:01.813691 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:01.814377 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:01.814380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:01.814381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:01.814385 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:01.814427 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:01.814442 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:01.814522 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:01.814589 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 08:47:01.814630 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":106,"threads_started":1}
I20260809 08:47:01.815444 31656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2: Bootstrap starting.
I20260809 08:47:01.815946 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:01.816246 31656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:01.816581 31656 log.cc:824] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:01.817385 31656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2: No bootstrap required, opened a new log
I20260809 08:47:01.818706 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:01.817622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50999 (local address 127.31.63.126:44355)
0809 08:47:01.817686 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:01.817688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:01.817873 (+ 185us) server_negotiation.cc:407] Connection header received
0809 08:47:01.817935 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:01.817936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:01.817940 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:01.817985 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:01.818485 (+ 500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:01.818486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:01.818486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:01.818489 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:01.818500 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:01.818504 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:01.818546 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:01.818608 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 08:47:01.818618 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:47:01.819927 31656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514421488c144d5a9d6ed0aa823b6e2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44355 } }
I20260809 08:47:01.820068 31656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:01.820123 31656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7514421488c144d5a9d6ed0aa823b6e2, State: Initialized, Role: FOLLOWER
I20260809 08:47:01.820248 31656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514421488c144d5a9d6ed0aa823b6e2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44355 } }
I20260809 08:47:01.820336 31656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:01.820360 31656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:01.820431 31656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:01.820957 31656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514421488c144d5a9d6ed0aa823b6e2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44355 } }
I20260809 08:47:01.821076 31656 leader_election.cc:302] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7514421488c144d5a9d6ed0aa823b6e2; no voters:
I20260809 08:47:01.821233 31656 leader_election.cc:288] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:01.821314 31674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:01.821482 31656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:01.821519 31674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [term 1 LEADER]: Becoming Leader. State: Replica: 7514421488c144d5a9d6ed0aa823b6e2, State: Running, Role: LEADER
I20260809 08:47:01.821694 31674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514421488c144d5a9d6ed0aa823b6e2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44355 } }
I20260809 08:47:01.822070 31674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7514421488c144d5a9d6ed0aa823b6e2. Latest consensus state: current_term: 1 leader_uuid: "7514421488c144d5a9d6ed0aa823b6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514421488c144d5a9d6ed0aa823b6e2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44355 } } }
I20260809 08:47:01.822104 31675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7514421488c144d5a9d6ed0aa823b6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7514421488c144d5a9d6ed0aa823b6e2" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 44355 } } }
I20260809 08:47:01.822207 31674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:01.822296 31675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:01.822770 31689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:01.823246 31689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:01.824702 31689 catalog_manager.cc:1347] Generated new cluster ID: d921b530dabd49a88b04cdcfc6e28459
I20260809 08:47:01.824766 31689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:01.833144 31689 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:01.833333 31689 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:01.846848 31689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7514421488c144d5a9d6ed0aa823b6e2: Generated new TSK 0
I20260809 08:47:01.847076 31689 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:01.852444 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 31552
2026-08-09T08:47:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1520 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:01 dist-test-slave-gkw7 krb5kdc[31708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:01 dist-test-slave-gkw7 krb5kdc[31708](info): set up 2 sockets
Aug 09 08:47:01 dist-test-slave-gkw7 krb5kdc[31708](info): commencing operation
krb5kdc: starting...
W20260809 08:47:03.066784 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:47:03 dist-test-slave-gkw7 krb5kdc[31708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:47:03.191625 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:32825
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:41987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:32825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:03.275457 31729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:03.275779 31729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:03.275821 31729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:03.277776 31729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:03.277853 31729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:03.277881 31729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:03.277910 31729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:03.277931 31729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:03.279836 31729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:41987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:32825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:32825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:03.280269 31729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:03.280571 31729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:03.286665 31746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:03.286768 31739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:03.286674 31740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:03.288010 31729 server_base.cc:1034] running on GCE node
I20260809 08:47:03.288280 31729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:03.288604 31729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:03.289750 31729 hybrid_clock.cc:648] HybridClock initialized: now 1786265223289737 us; error 28 us; skew 500 ppm
Aug 09 08:47:03 dist-test-slave-gkw7 krb5kdc[31708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265223, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:03.292260 31729 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:47:03.292733 31729 webserver.cc:460] Webserver started at http://127.31.63.126:36585/ using document root <none> and password file <none>
I20260809 08:47:03.292943 31729 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:03.292984 31729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:03.293077 31729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:03.294178 31729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "eca615f905404e1ea617c2e3ec032b24"
format_stamp: "Formatted at 2026-08-09 08:47:03 on dist-test-slave-gkw7"
server_key: "463d3a7a67de9bc3ea3199b8568876f7"
server_key_iv: "b8f0583101bb538feb82822f0f126e66"
server_key_version: "encryptionkey@0"
I20260809 08:47:03.294488 31729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "eca615f905404e1ea617c2e3ec032b24"
format_stamp: "Formatted at 2026-08-09 08:47:03 on dist-test-slave-gkw7"
server_key: "463d3a7a67de9bc3ea3199b8568876f7"
server_key_iv: "b8f0583101bb538feb82822f0f126e66"
server_key_version: "encryptionkey@0"
I20260809 08:47:03.295766 31729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:03.296496 31754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:03.296687 31729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:03.296775 31729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "eca615f905404e1ea617c2e3ec032b24"
format_stamp: "Formatted at 2026-08-09 08:47:03 on dist-test-slave-gkw7"
server_key: "463d3a7a67de9bc3ea3199b8568876f7"
server_key_iv: "b8f0583101bb538feb82822f0f126e66"
server_key_version: "encryptionkey@0"
I20260809 08:47:03.296833 31729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:03.317075 31729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:03.319293 31729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:03.319489 31729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:03.324512 31729 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:32825
I20260809 08:47:03.324543 31836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:32825 every 8 connection(s)
I20260809 08:47:03.324980 31729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:03.325937 31837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:03.326959 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 31729
I20260809 08:47:03.327085 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:47:03.327258 31997 external_mini_cluster.cc:1428] Setting key 6c1710504df4b1e9c01bb3927ca25cdd
I20260809 08:47:03.329241 31837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24: Bootstrap starting.
I20260809 08:47:03.329990 31837 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:03.330317 31837 log.cc:824] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:03.331066 31837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24: No bootstrap required, opened a new log
I20260809 08:47:03.333446 31837 raft_consensus.cc:348] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca615f905404e1ea617c2e3ec032b24" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32825 } }
I20260809 08:47:03.333643 31837 raft_consensus.cc:374] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:03.333673 31837 raft_consensus.cc:729] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca615f905404e1ea617c2e3ec032b24, State: Initialized, Role: FOLLOWER
I20260809 08:47:03.333817 31837 consensus_queue.cc:260] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca615f905404e1ea617c2e3ec032b24" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32825 } }
I20260809 08:47:03.333895 31837 raft_consensus.cc:388] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:03.333918 31837 raft_consensus.cc:482] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:03.333981 31837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 08:47:03 dist-test-slave-gkw7 krb5kdc[31708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:47:03.334580 31837 raft_consensus.cc:504] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca615f905404e1ea617c2e3ec032b24" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32825 } }
I20260809 08:47:03.334707 31837 leader_election.cc:302] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eca615f905404e1ea617c2e3ec032b24; no voters:
I20260809 08:47:03.334901 31837 leader_election.cc:288] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:03.335088 31846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:03.335176 31837 sys_catalog.cc:564] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:03.335251 31846 raft_consensus.cc:686] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [term 1 LEADER]: Becoming Leader. State: Replica: eca615f905404e1ea617c2e3ec032b24, State: Running, Role: LEADER
I20260809 08:47:03.335338 31846 consensus_queue.cc:237] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca615f905404e1ea617c2e3ec032b24" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32825 } }
I20260809 08:47:03.335675 31846 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eca615f905404e1ea617c2e3ec032b24. Latest consensus state: current_term: 1 leader_uuid: "eca615f905404e1ea617c2e3ec032b24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca615f905404e1ea617c2e3ec032b24" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32825 } } }
I20260809 08:47:03.335804 31846 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:03.335903 31848 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eca615f905404e1ea617c2e3ec032b24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca615f905404e1ea617c2e3ec032b24" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 32825 } } }
I20260809 08:47:03.336090 31857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:03.337049 31857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:03.338686 31857 catalog_manager.cc:1347] Generated new cluster ID: 4963ad497d2c48dc8c93b2944372d21e
I20260809 08:47:03.338742 31857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:03.336076 31848 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:03.339809 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:03.327777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42733 (local address 127.31.63.126:32825)
0809 08:47:03.327973 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:03.327982 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:03.328017 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:47:03.328482 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:03.328491 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:03.328555 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:03.328700 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:03.329392 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:03.330768 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:03.331593 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:03.332223 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:03.332341 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:03.334947 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:03.334951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:03.334952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:03.334964 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:03.338282 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:03.338659 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:03.338662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:03.338663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:03.338795 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:03.339012 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:03.339013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:03.339013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:03.339234 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:03.339378 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:03.339486 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 08:47:03.339644 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260809 08:47:03.342581 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:03.350340 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:03.344137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39159 (local address 127.31.63.126:32825)
0809 08:47:03.344172 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:03.344174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:03.344355 (+ 181us) server_negotiation.cc:407] Connection header received
0809 08:47:03.344401 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:03.344402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:03.344411 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:03.344466 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:03.344992 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:03.345471 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:03.346204 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:03.346442 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:03.346521 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:03.347711 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:03.347712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:03.347712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:03.347719 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:03.349144 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:03.349502 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:03.349503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:03.349504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:03.349617 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:03.349827 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:03.349828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:03.349828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:03.349939 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:03.350055 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:03.350167 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 08:47:03.350244 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:47:03.373575 31857 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:03.374154 31857 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:03.378700 31857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eca615f905404e1ea617c2e3ec032b24: Generated new TSK 0
I20260809 08:47:03.378906 31857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:03.381434 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 31729
2026-08-09T08:47:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1526 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:03 dist-test-slave-gkw7 krb5kdc[31881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:03 dist-test-slave-gkw7 krb5kdc[31881](info): set up 2 sockets
Aug 09 08:47:03 dist-test-slave-gkw7 krb5kdc[31881](info): commencing operation
krb5kdc: starting...
W20260809 08:47:04.591981 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:47:04 dist-test-slave-gkw7 krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:47:04.716063 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:39723
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:43493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:39723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:04.800405 31898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:04.800633 31898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:04.800669 31898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:04.801864 31898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:04.801925 31898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:04.801949 31898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:04.801981 31898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:04.802006 31898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:04.803609 31898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:43493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:39723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:39723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:04.803905 31898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:04.804154 31898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:04.809201 31913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:04.809322 31910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:04.809146 31911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:04.809394 31898 server_base.cc:1034] running on GCE node
I20260809 08:47:04.809751 31898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:04.810067 31898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:04.811228 31898 hybrid_clock.cc:648] HybridClock initialized: now 1786265224811212 us; error 28 us; skew 500 ppm
Aug 09 08:47:04 dist-test-slave-gkw7 krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265224, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:04.817989 31898 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:47:04.818415 31898 webserver.cc:460] Webserver started at http://127.31.63.126:34719/ using document root <none> and password file <none>
I20260809 08:47:04.818665 31898 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:04.818737 31898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:04.818892 31898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:04.819912 31898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "a092a155090a49b9ad149a038fed6b58"
format_stamp: "Formatted at 2026-08-09 08:47:04 on dist-test-slave-gkw7"
server_key: "fe2048fe8bb5f666a31e81335da0fb3b"
server_key_iv: "a73e1210daf849fc1ca507eb4795671f"
server_key_version: "encryptionkey@0"
I20260809 08:47:04.820273 31898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "a092a155090a49b9ad149a038fed6b58"
format_stamp: "Formatted at 2026-08-09 08:47:04 on dist-test-slave-gkw7"
server_key: "fe2048fe8bb5f666a31e81335da0fb3b"
server_key_iv: "a73e1210daf849fc1ca507eb4795671f"
server_key_version: "encryptionkey@0"
I20260809 08:47:04.821661 31898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:04.822419 31928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:04.822590 31898 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:04.822691 31898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "a092a155090a49b9ad149a038fed6b58"
format_stamp: "Formatted at 2026-08-09 08:47:04 on dist-test-slave-gkw7"
server_key: "fe2048fe8bb5f666a31e81335da0fb3b"
server_key_iv: "a73e1210daf849fc1ca507eb4795671f"
server_key_version: "encryptionkey@0"
I20260809 08:47:04.822784 31898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:04.840186 31898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:04.842202 31898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:04.842372 31898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:04.846585 31898 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:39723
I20260809 08:47:04.846621 32018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:39723 every 8 connection(s)
I20260809 08:47:04.846992 31898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:04.847862 32019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:04.850310 32019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58: Bootstrap starting.
I20260809 08:47:04.850910 32019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:04.851231 32019 log.cc:824] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:04.851502 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 31898
I20260809 08:47:04.851629 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:47:04.851812 31997 external_mini_cluster.cc:1428] Setting key d40a62d4a19fdc4c8934ab19778ad111
I20260809 08:47:04.852039 32019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58: No bootstrap required, opened a new log
I20260809 08:47:04.853917 32019 raft_consensus.cc:348] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a092a155090a49b9ad149a038fed6b58" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39723 } }
I20260809 08:47:04.854077 32019 raft_consensus.cc:374] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:04.854142 32019 raft_consensus.cc:729] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a092a155090a49b9ad149a038fed6b58, State: Initialized, Role: FOLLOWER
I20260809 08:47:04.854317 32019 consensus_queue.cc:260] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a092a155090a49b9ad149a038fed6b58" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39723 } }
I20260809 08:47:04.854416 32019 raft_consensus.cc:388] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:04.854462 32019 raft_consensus.cc:482] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:04.854509 32019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:04.855064 32019 raft_consensus.cc:504] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a092a155090a49b9ad149a038fed6b58" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39723 } }
I20260809 08:47:04.855196 32019 leader_election.cc:302] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a092a155090a49b9ad149a038fed6b58; no voters:
I20260809 08:47:04.855376 32019 leader_election.cc:288] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:04.855405 32025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:04.855584 32025 raft_consensus.cc:686] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [term 1 LEADER]: Becoming Leader. State: Replica: a092a155090a49b9ad149a038fed6b58, State: Running, Role: LEADER
I20260809 08:47:04.855676 32019 sys_catalog.cc:564] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:04.855777 32025 consensus_queue.cc:237] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a092a155090a49b9ad149a038fed6b58" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39723 } }
I20260809 08:47:04.856202 32025 sys_catalog.cc:455] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a092a155090a49b9ad149a038fed6b58. Latest consensus state: current_term: 1 leader_uuid: "a092a155090a49b9ad149a038fed6b58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a092a155090a49b9ad149a038fed6b58" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39723 } } }
I20260809 08:47:04.856415 32025 sys_catalog.cc:458] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:04.856350 32027 sys_catalog.cc:455] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a092a155090a49b9ad149a038fed6b58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a092a155090a49b9ad149a038fed6b58" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 39723 } } }
I20260809 08:47:04.856644 32027 sys_catalog.cc:458] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:04.857132 32036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:04.857813 32036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:04.859416 32036 catalog_manager.cc:1347] Generated new cluster ID: f37354bb8c364fda92bd6d72c8b1cc8d
I20260809 08:47:04.859467 32036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:04.859455 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:04.852313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56713 (local address 127.31.63.126:39723)
0809 08:47:04.852492 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:04.852503 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:04.852535 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:47:04.853896 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:04.853904 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:04.853951 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:04.854093 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:04.854369 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:04.855845 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:04.857037 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:04.857804 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:04.857914 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:04.858648 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:04.858651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:04.858652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:04.858868 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:04.858907 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:04.858921 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:04.858987 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:04.859066 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 08:47:04.859239 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260809 08:47:04.866570 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:04.872751 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:04.869867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36365 (local address 127.31.63.126:39723)
0809 08:47:04.869902 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:04.869905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:04.870098 (+ 193us) server_negotiation.cc:407] Connection header received
0809 08:47:04.870177 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:04.870178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:04.870189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:04.870245 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:04.870378 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:04.870848 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:04.871549 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:04.871755 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:04.871831 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:04.872519 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:04.872520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:04.872521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:04.872527 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:04.872541 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:04.872545 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:04.872589 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:04.872650 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 08:47:04.872671 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:47:04.876658 32036 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:04.877382 32036 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:04.883953 32036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a092a155090a49b9ad149a038fed6b58: Generated new TSK 0
I20260809 08:47:04.884124 32036 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:04.891463 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 31898
2026-08-09T08:47:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:04 dist-test-slave-gkw7 krb5kdc[32060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:04 dist-test-slave-gkw7 krb5kdc[32060](info): set up 2 sockets
Aug 09 08:47:04 dist-test-slave-gkw7 krb5kdc[32060](info): commencing operation
krb5kdc: starting...
W20260809 08:47:06.110229 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:47:06 dist-test-slave-gkw7 krb5kdc[32060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:47:06.249054 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:36901
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:46175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:36901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:06.335691 32079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:06.335935 32079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:06.335974 32079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:06.337102 32079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:06.337182 32079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:06.337209 32079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:06.337240 32079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:06.337263 32079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:06.338737 32079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:36901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:36901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:06.339030 32079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:06.339267 32079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:06.344339 32087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:06.344437 32089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:06.344338 32092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:06.344491 32079 server_base.cc:1034] running on GCE node
I20260809 08:47:06.344830 32079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:06.345118 32079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:06.346297 32079 hybrid_clock.cc:648] HybridClock initialized: now 1786265226346280 us; error 47 us; skew 500 ppm
Aug 09 08:47:06 dist-test-slave-gkw7 krb5kdc[32060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265226, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:06.348708 32079 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:47:06.349125 32079 webserver.cc:460] Webserver started at http://127.31.63.126:42107/ using document root <none> and password file <none>
I20260809 08:47:06.349345 32079 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:06.349412 32079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:06.349606 32079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:06.350719 32079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "9116d4527398432fb11b44e8bd66851d"
format_stamp: "Formatted at 2026-08-09 08:47:06 on dist-test-slave-gkw7"
server_key: "a31cc8fc9dbd7360e6bdd76094afa885"
server_key_iv: "df2c20809243d39e1b4f54cb082009d2"
server_key_version: "encryptionkey@0"
I20260809 08:47:06.351102 32079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "9116d4527398432fb11b44e8bd66851d"
format_stamp: "Formatted at 2026-08-09 08:47:06 on dist-test-slave-gkw7"
server_key: "a31cc8fc9dbd7360e6bdd76094afa885"
server_key_iv: "df2c20809243d39e1b4f54cb082009d2"
server_key_version: "encryptionkey@0"
I20260809 08:47:06.352433 32079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:06.353255 32100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:06.353461 32079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:06.353648 32079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "9116d4527398432fb11b44e8bd66851d"
format_stamp: "Formatted at 2026-08-09 08:47:06 on dist-test-slave-gkw7"
server_key: "a31cc8fc9dbd7360e6bdd76094afa885"
server_key_iv: "df2c20809243d39e1b4f54cb082009d2"
server_key_version: "encryptionkey@0"
I20260809 08:47:06.353750 32079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:06.357483 32079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:06.359622 32079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:06.359827 32079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:06.364588 32079 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:36901
I20260809 08:47:06.364652 32174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:36901 every 8 connection(s)
I20260809 08:47:06.365013 32079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:06.365968 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 32079
I20260809 08:47:06.366130 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:47:06.366155 32176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:06.366320 31997 external_mini_cluster.cc:1428] Setting key 8936e2d6b797594acc97fd4abe8582af
I20260809 08:47:06.368896 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:06.366898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43921 (local address 127.31.63.126:36901)
0809 08:47:06.367085 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:06.367093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:06.367125 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:47:06.367591 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:06.367598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:06.367623 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:06.367764 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:06.368446 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:06.368450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:06.368450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:06.368454 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:06.368491 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:06.368501 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:06.368555 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:06.368641 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 08:47:06.368672 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260809 08:47:06.369524 32176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d: Bootstrap starting.
I20260809 08:47:06.369894 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:06.370289 32176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:06.370616 32176 log.cc:824] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:06.371495 32176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d: No bootstrap required, opened a new log
I20260809 08:47:06.372543 32181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:06.371576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45269 (local address 127.31.63.126:36901)
0809 08:47:06.371614 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:06.371616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:06.371712 (+ 96us) server_negotiation.cc:407] Connection header received
0809 08:47:06.371774 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:06.371775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:06.371779 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:06.371821 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:06.372334 (+ 513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:06.372335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:06.372335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:06.372339 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:06.372350 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:06.372354 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:06.372397 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:06.372450 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 08:47:06.372460 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:47:06.373914 32176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9116d4527398432fb11b44e8bd66851d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36901 } }
I20260809 08:47:06.374045 32176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:06.374068 32176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9116d4527398432fb11b44e8bd66851d, State: Initialized, Role: FOLLOWER
I20260809 08:47:06.374212 32176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9116d4527398432fb11b44e8bd66851d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36901 } }
I20260809 08:47:06.374322 32176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:06.374356 32176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:06.374397 32176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:06.374927 32176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9116d4527398432fb11b44e8bd66851d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36901 } }
I20260809 08:47:06.375037 32176 leader_election.cc:302] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9116d4527398432fb11b44e8bd66851d; no voters:
I20260809 08:47:06.375205 32176 leader_election.cc:288] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:06.375295 32192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:06.375452 32192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [term 1 LEADER]: Becoming Leader. State: Replica: 9116d4527398432fb11b44e8bd66851d, State: Running, Role: LEADER
I20260809 08:47:06.375452 32176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:06.375550 32192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9116d4527398432fb11b44e8bd66851d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36901 } }
I20260809 08:47:06.375929 32193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9116d4527398432fb11b44e8bd66851d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9116d4527398432fb11b44e8bd66851d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36901 } } }
I20260809 08:47:06.375934 32195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9116d4527398432fb11b44e8bd66851d. Latest consensus state: current_term: 1 leader_uuid: "9116d4527398432fb11b44e8bd66851d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9116d4527398432fb11b44e8bd66851d" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 36901 } } }
I20260809 08:47:06.376041 32193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:06.376048 32195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:06.376266 32204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:06.376775 32204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:06.378368 32204 catalog_manager.cc:1347] Generated new cluster ID: f2f4d7ec89e1426cb8b9a31833a7f23b
I20260809 08:47:06.378409 32204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:06.383437 32204 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:06.383595 32204 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:06.389735 32204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9116d4527398432fb11b44e8bd66851d: Generated new TSK 0
I20260809 08:47:06.389918 32204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:06.392902 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 32079
2026-08-09T08:47:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1502 ms)
[----------] 18 tests from AuthTokenIssuingTest (27965 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:06 dist-test-slave-gkw7 krb5kdc[32224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:06 dist-test-slave-gkw7 krb5kdc[32224](info): set up 2 sockets
Aug 09 08:47:06 dist-test-slave-gkw7 krb5kdc[32224](info): commencing operation
krb5kdc: starting...
W20260809 08:47:07.634052 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:47:07 dist-test-slave-gkw7 krb5kdc[32224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:47:07.764654 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33301
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33301,127.31.63.125:40279,127.31.63.124:46297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:07.847411 32240 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:47:07.847616 32240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:07.847635 32240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:07.847654 32240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:07.848698 32240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:07.848731 32240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:07.848745 32240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:07.848758 32240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:07.848769 32240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:07.850286 32240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33301,127.31.63.125:40279,127.31.63.124:46297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:33301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:07.850487 32240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:07.850654 32240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:07.855858 32251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:07.855854 32246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:07.855883 32240 server_base.cc:1034] running on GCE node
W20260809 08:47:07.855849 32247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:07.856315 32240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:07.856683 32240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:07.857878 32240 hybrid_clock.cc:648] HybridClock initialized: now 1786265227857862 us; error 27 us; skew 500 ppm
Aug 09 08:47:07 dist-test-slave-gkw7 krb5kdc[32224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265227, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:07.860441 32240 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:47:07.860908 32240 webserver.cc:460] Webserver started at http://127.31.63.126:39637/ using document root <none> and password file <none>
I20260809 08:47:07.861145 32240 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:07.861220 32240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:07.861358 32240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:07.862517 32240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "2f0f322020614aceb561c34d5a05aa42"
format_stamp: "Formatted at 2026-08-09 08:47:07 on dist-test-slave-gkw7"
server_key: "e50dfc284f8ed3181427564b54f7b827"
server_key_iv: "385674cff7f44e9313d5a0e41cadb7b0"
server_key_version: "encryptionkey@0"
I20260809 08:47:07.862908 32240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "2f0f322020614aceb561c34d5a05aa42"
format_stamp: "Formatted at 2026-08-09 08:47:07 on dist-test-slave-gkw7"
server_key: "e50dfc284f8ed3181427564b54f7b827"
server_key_iv: "385674cff7f44e9313d5a0e41cadb7b0"
server_key_version: "encryptionkey@0"
I20260809 08:47:07.864286 32240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:07.865036 32260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:07.865216 32240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:07.865320 32240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "2f0f322020614aceb561c34d5a05aa42"
format_stamp: "Formatted at 2026-08-09 08:47:07 on dist-test-slave-gkw7"
server_key: "e50dfc284f8ed3181427564b54f7b827"
server_key_iv: "385674cff7f44e9313d5a0e41cadb7b0"
server_key_version: "encryptionkey@0"
I20260809 08:47:07.865404 32240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:07.886543 32240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:07.888513 32240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:07.888728 32240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:07.892972 32240 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:33301
I20260809 08:47:07.893049 32335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:33301 every 8 connection(s)
I20260809 08:47:07.893419 32240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:07.894497 32338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:07.896099 32338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:07.900266 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 32240
I20260809 08:47:07.900365 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:47:07.900535 31997 external_mini_cluster.cc:1428] Setting key cf27d60265a4f9323e0d7c617edd920d
Aug 09 08:47:07 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265227, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:47:07.906517 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:07.896797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33552 (local address 127.31.63.126:33301)
0809 08:47:07.896976 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:07.896980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:07.897067 (+ 87us) server_negotiation.cc:407] Connection header received
0809 08:47:07.897629 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:07.897631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:07.897660 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:07.897739 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:07.898343 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:07.899240 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:07.900136 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:07.900602 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:07.900734 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:07.903156 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:07.903166 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:07.903166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:07.903175 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:07.904853 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:07.905343 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:07.905346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:07.905347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:07.905425 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:07.905719 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:07.905720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:07.905720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:07.905928 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:07.905989 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:07.906253 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 08:47:07.906350 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":122,"threads_started":1}
I20260809 08:47:07.906737 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:07.896737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33301 (local address 127.0.0.1:33552)
0809 08:47:07.896913 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:07.896964 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:07.897444 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:07.897768 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:07.897771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:07.898023 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:07.898210 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:07.898214 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:07.899328 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:07.899329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:07.900003 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:07.900005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:07.900739 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:07.900740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:07.900816 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:07.901418 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:07.901426 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:07.903067 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:07.904944 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:07.904951 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:07.904953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:07.905252 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:07.905501 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:07.905502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:07.905503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:07.905641 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:07.906004 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:07.906013 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:07.906098 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 08:47:07.906217 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 08:47:07.906466 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":64,"thread_start_us":74,"threads_started":1}
I20260809 08:47:07.907538 32338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:07.907919 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:07.907778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:40279 (local address 127.0.0.1:54414)
0809 08:47:07.907833 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:07.907896 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:40279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:47:07.908085 32266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.63.125:40279: connect: Connection refused (error 111)
I20260809 08:47:07.908612 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:07.908504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:40279 (local address 127.0.0.1:54428)
0809 08:47:07.908553 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:07.908583 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:40279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 08:47:07.908741 32338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.125:40279: Network error: Client connection negotiation failed: client connection to 127.31.63.125:40279: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.63.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab.
Entry for principal kudu/127.31.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab.
I20260809 08:47:07.928786 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.125:40279
--webserver_interface=127.31.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab
--principal=kudu/127.31.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33301,127.31.63.125:40279,127.31.63.124:46297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:47:07.957994 32338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } attempt: 1
I20260809 08:47:07.958572 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:07.958395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:40279 (local address 127.0.0.1:54436)
0809 08:47:07.958477 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:07.958531 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:40279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 08:47:07.958981 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:07.958870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:40279 (local address 127.0.0.1:54450)
0809 08:47:07.958925 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:07.958958 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:40279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 08:47:07.959111 32338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.125:40279: Network error: Client connection negotiation failed: client connection to 127.31.63.125:40279: connect: Connection refused (error 111)
W20260809 08:47:08.012279 32349 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:47:08.012557 32349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:08.012588 32349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:08.012627 32349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:08.013808 32349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:08.013880 32349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:08.013904 32349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:08.013926 32349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:08.013957 32349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:08.015450 32349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33301,127.31.63.125:40279,127.31.63.124:46297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab
--principal=kudu/127.31.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.125:40279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:08.015730 32349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:08.015990 32349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:08.021160 32355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:08.021273 32349 server_base.cc:1034] running on GCE node
W20260809 08:47:08.021092 32356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:08.021092 32361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:08.021669 32349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:08.022014 32349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:08.023211 32349 hybrid_clock.cc:648] HybridClock initialized: now 1786265228023201 us; error 44 us; skew 500 ppm
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265228, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:08.026044 32349 init.cc:377] Logged in from keytab as kudu/127.31.63.125@KRBTEST.COM (short username kudu)
I20260809 08:47:08.026436 32349 webserver.cc:460] Webserver started at http://127.31.63.125:43699/ using document root <none> and password file <none>
I20260809 08:47:08.026641 32349 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:08.026681 32349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:08.026777 32349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:08.027263 32338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } attempt: 2
I20260809 08:47:08.027690 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.027542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:40279 (local address 127.0.0.1:54462)
0809 08:47:08.027614 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.027655 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:40279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 08:47:08.028098 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.027989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:40279 (local address 127.0.0.1:54474)
0809 08:47:08.028045 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.028075 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:40279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 08:47:08.027956 32349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/data/instance:
uuid: "048a4935d60e41779be0af14354852dc"
format_stamp: "Formatted at 2026-08-09 08:47:08 on dist-test-slave-gkw7"
server_key: "b9d0f6e9b3bcea01376b84c7689ef1bb"
server_key_iv: "e0bbde4a91ef2b21f6158e955a19efc4"
server_key_version: "encryptionkey@0"
W20260809 08:47:08.028224 32338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.125:40279: Network error: Client connection negotiation failed: client connection to 127.31.63.125:40279: connect: Connection refused (error 111)
I20260809 08:47:08.028362 32349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/wal/instance:
uuid: "048a4935d60e41779be0af14354852dc"
format_stamp: "Formatted at 2026-08-09 08:47:08 on dist-test-slave-gkw7"
server_key: "b9d0f6e9b3bcea01376b84c7689ef1bb"
server_key_iv: "e0bbde4a91ef2b21f6158e955a19efc4"
server_key_version: "encryptionkey@0"
I20260809 08:47:08.029821 32349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:08.030596 32368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:08.030834 32349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:08.030933 32349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/wal
uuid: "048a4935d60e41779be0af14354852dc"
format_stamp: "Formatted at 2026-08-09 08:47:08 on dist-test-slave-gkw7"
server_key: "b9d0f6e9b3bcea01376b84c7689ef1bb"
server_key_iv: "e0bbde4a91ef2b21f6158e955a19efc4"
server_key_version: "encryptionkey@0"
I20260809 08:47:08.031013 32349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:08.046259 32349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:08.048437 32349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:08.048604 32349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:08.053340 32349 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.125:40279
I20260809 08:47:08.053395 32447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.125:40279 every 8 connection(s)
I20260809 08:47:08.053819 32349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
I20260809 08:47:08.054736 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 32349
I20260809 08:47:08.054854 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-1/wal/instance
I20260809 08:47:08.055037 31997 external_mini_cluster.cc:1428] Setting key 93fadcc39996c02b1d41aeed42b4db91
I20260809 08:47:08.055125 32449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:08.056993 32449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.63.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.124@KRBTEST.COM" created.
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265228, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:47:08.070974 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.057869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33301 (local address 127.0.0.1:33566)
0809 08:47:08.058090 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.058132 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:08.058707 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:08.059011 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:08.059015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:08.059305 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:08.059533 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.059537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.060352 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.060354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.061328 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.061331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.061902 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.061904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.062060 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.062667 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:08.062676 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:08.067082 (+ 4406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:08.069267 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.069276 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.069279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.069660 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.069885 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.069886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.069886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.070036 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.070409 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:08.070416 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:08.070552 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 08:47:08.070644 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:47:08.070785 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":111,"threads_started":1}
I20260809 08:47:08.070973 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.057897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33566 (local address 127.31.63.126:33301)
0809 08:47:08.057962 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.057964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.058242 (+ 278us) server_negotiation.cc:407] Connection header received
0809 08:47:08.058838 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.058839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.058860 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.058926 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.059654 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.060233 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.061428 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.061749 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.061870 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.067240 (+ 5370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.067242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.067242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.067253 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.069166 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.069753 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.069754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.069755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.069818 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.070113 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.070114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.070115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.070331 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.070415 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.070699 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.070808 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:47:08.071784 32449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265228, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
I20260809 08:47:08.079815 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.072083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:40279 (local address 127.0.0.1:54478)
0809 08:47:08.072132 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.072143 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:08.072186 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:08.072505 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:08.072506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:08.072651 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:08.072702 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.072703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.073591 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.073592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.074118 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.074120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.074636 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.074637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.074706 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.075152 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:08.075158 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:08.076644 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:08.078648 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.078650 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.078650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.078888 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.079137 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.079137 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.079138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.079197 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.079561 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:08.079562 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:08.079603 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 08:47:08.079659 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 08:47:08.079743 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 08:47:08.079952 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.072170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54478 (local address 127.31.63.125:40279)
0809 08:47:08.072359 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.072364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.072382 (+ 18us) server_negotiation.cc:407] Connection header received
0809 08:47:08.072408 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.072410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.072421 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.072504 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.072780 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.073474 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.074215 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.074547 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.074627 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.076730 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.076737 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.076738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.076744 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.078387 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.078977 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.078980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.078980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.079034 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.079262 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.079262 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.079263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.079468 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.079582 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.079685 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.079797 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":160,"thread_start_us":129,"threads_started":1}
I20260809 08:47:08.080612 32449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:08.080999 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.080875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56728)
0809 08:47:08.080922 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.080980 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 08:47:08.081143 32372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
I20260809 08:47:08.081779 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.081514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56736)
0809 08:47:08.081715 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.081754 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
W20260809 08:47:08.081951 32449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:46297: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab.
Entry for principal kudu/127.31.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab.
I20260809 08:47:08.085471 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.124:46297
--webserver_interface=127.31.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab
--principal=kudu/127.31.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:33301,127.31.63.125:40279,127.31.63.124:46297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:47:08.119396 32338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } attempt: 3
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265227, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
I20260809 08:47:08.127508 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.119849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:40279 (local address 127.0.0.1:54484)
0809 08:47:08.120030 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.120046 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:08.120122 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:08.120334 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:08.120335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:08.120532 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:08.120606 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.120608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.121383 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.121385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.122091 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.122093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.122592 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.122593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.122694 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.123235 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:08.123241 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:08.124589 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:08.126336 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.126337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.126338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.126537 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.126766 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.126767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.126767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.126840 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.127129 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:08.127130 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:08.127184 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 08:47:08.127269 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:47:08.127385 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260809 08:47:08.127508 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.120077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54484 (local address 127.31.63.125:40279)
0809 08:47:08.120148 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.120151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.120162 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:47:08.120228 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.120231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.120250 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.120325 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.120716 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.121287 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.122220 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.122467 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.122592 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.124707 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.124708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.124709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.124717 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.126239 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.126620 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.126622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.126622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.126688 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.126914 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.126915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.126916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.127043 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.127125 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.127315 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.127402 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:47:08.128180 32338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:08.128557 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.128453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56742)
0809 08:47:08.128504 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.128537 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 08:47:08.129338 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.129225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56750)
0809 08:47:08.129284 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.129313 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 08:47:08.129446 32338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:46297: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
I20260809 08:47:08.131181 32449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } attempt: 1
I20260809 08:47:08.131556 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.131434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56764)
0809 08:47:08.131491 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.131520 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:47:08.131881 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.131782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56776)
0809 08:47:08.131830 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.131856 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 08:47:08.131994 32449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:46297: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
I20260809 08:47:08.160609 32338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } attempt: 1
I20260809 08:47:08.161181 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.161031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56778)
0809 08:47:08.161105 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.161149 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 08:47:08.161671 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.161546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56786)
0809 08:47:08.161612 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.161648 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 08:47:08.161844 32338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:46297: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
W20260809 08:47:08.173352 32456 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:47:08.173626 32456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:08.173655 32456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:08.173681 32456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:08.174772 32456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:08.174805 32456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:08.174820 32456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:08.174832 32456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:08.174844 32456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:08.176275 32456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:33301,127.31.63.125:40279,127.31.63.124:46297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab
--principal=kudu/127.31.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.124:46297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:08.176470 32456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:08.176643 32456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:08.181618 32467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:08.181646 32464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:08.181633 32465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:08.182034 32456 server_base.cc:1034] running on GCE node
I20260809 08:47:08.182299 32456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:08.182583 32456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:08.183756 32456 hybrid_clock.cc:648] HybridClock initialized: now 1786265228183737 us; error 30 us; skew 500 ppm
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265228, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:08.186390 32456 init.cc:377] Logged in from keytab as kudu/127.31.63.124@KRBTEST.COM (short username kudu)
I20260809 08:47:08.186878 32456 webserver.cc:460] Webserver started at http://127.31.63.124:46253/ using document root <none> and password file <none>
I20260809 08:47:08.187122 32456 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:08.187193 32456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:08.187350 32456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:08.188426 32456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/data/instance:
uuid: "3d0ffef8759b42a18517608fa7f87788"
format_stamp: "Formatted at 2026-08-09 08:47:08 on dist-test-slave-gkw7"
server_key: "4904d7502de9d164f286dd24f17e6369"
server_key_iv: "0c1adf8293503a62a65eb57b099cd054"
server_key_version: "encryptionkey@0"
I20260809 08:47:08.188799 32456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/wal/instance:
uuid: "3d0ffef8759b42a18517608fa7f87788"
format_stamp: "Formatted at 2026-08-09 08:47:08 on dist-test-slave-gkw7"
server_key: "4904d7502de9d164f286dd24f17e6369"
server_key_iv: "0c1adf8293503a62a65eb57b099cd054"
server_key_version: "encryptionkey@0"
I20260809 08:47:08.190336 32456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:08.191133 32480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:08.191330 32456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:08.191442 32456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/wal
uuid: "3d0ffef8759b42a18517608fa7f87788"
format_stamp: "Formatted at 2026-08-09 08:47:08 on dist-test-slave-gkw7"
server_key: "4904d7502de9d164f286dd24f17e6369"
server_key_iv: "0c1adf8293503a62a65eb57b099cd054"
server_key_version: "encryptionkey@0"
I20260809 08:47:08.191517 32456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:08.200130 32449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } attempt: 2
I20260809 08:47:08.200548 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.200420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56802)
0809 08:47:08.200482 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.200520 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:47:08.200892 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.200796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56814)
0809 08:47:08.200844 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.200868 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 08:47:08.200980 32449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:46297: Network error: Client connection negotiation failed: client connection to 127.31.63.124:46297: connect: Connection refused (error 111)
I20260809 08:47:08.206041 32456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:08.208415 32456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:08.208586 32456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:08.213050 32456 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.124:46297
I20260809 08:47:08.213112 32563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.124:46297 every 8 connection(s)
I20260809 08:47:08.213498 32456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
I20260809 08:47:08.214730 32565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:08.216385 32565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:08.221711 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 32456
I20260809 08:47:08.221807 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786264918773806-31997-0/minicluster-data/master-2/wal/instance
I20260809 08:47:08.221997 31997 external_mini_cluster.cc:1428] Setting key 632efd7a07c3fb4ed8acf70edb544943
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265228, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:47:08.226253 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.217145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33576 (local address 127.31.63.126:33301)
0809 08:47:08.217240 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.217242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.217588 (+ 346us) server_negotiation.cc:407] Connection header received
0809 08:47:08.218142 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.218143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.218155 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.218221 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.218856 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.219324 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.220362 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.220547 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.220605 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.223225 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.223227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.223228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.223237 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.224807 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.225188 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.225189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.225189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.225253 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.225517 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.225518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.225518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.225688 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.225741 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.226046 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.226131 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 08:47:08.226365 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.217123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:33301 (local address 127.0.0.1:33576)
0809 08:47:08.217372 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.217425 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:08.218046 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:08.218297 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:08.218302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:08.218553 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:08.218764 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.218768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.219401 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.219402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.220280 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.220283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.220632 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.220632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.220762 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.221318 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:08.221327 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:08.223092 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:08.224887 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.224893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.224894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.225113 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.225322 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.225322 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.225323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.225422 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.225752 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:08.225757 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:08.225901 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 08:47:08.226017 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 08:47:08.226155 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":126,"threads_started":1}
I20260809 08:47:08.226917 32565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265228, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
I20260809 08:47:08.232192 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.222439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56820 (local address 127.31.63.124:46297)
0809 08:47:08.222609 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.222613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.222671 (+ 58us) server_negotiation.cc:407] Connection header received
0809 08:47:08.222745 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.222758 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.222783 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.222860 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.223635 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.224504 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.225471 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.225903 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.225978 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.228801 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.228809 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.228810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.228817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.230709 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.231125 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.231128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.231128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.231190 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.231603 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.231604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.231604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.231745 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.231924 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.231996 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.232075 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260809 08:47:08.234045 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.227199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:40279 (local address 127.0.0.1:54486)
0809 08:47:08.227258 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.227275 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:08.227304 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:08.227468 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:08.227468 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:08.227576 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:08.227622 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.227623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.228325 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.228326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.228881 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.228883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.229254 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.229255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.229327 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.229737 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:08.229741 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:08.231091 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:08.232848 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.232849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.232850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.233097 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.233328 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.233329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.233329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.233409 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.233741 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:08.233741 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:08.233777 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 08:47:08.233861 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:47:08.233953 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 08:47:08.234177 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.227286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54486 (local address 127.31.63.125:40279)
0809 08:47:08.227339 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.227340 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.227350 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:08.227380 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.227381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.227391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.227469 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.227692 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.228238 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.228968 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.229173 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.229256 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.231179 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.231181 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.231181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.231188 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.232754 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.233187 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.233188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.233189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.233256 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.233486 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.233488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.233488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.233668 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.233743 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.233903 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.234021 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:47:08.234512 32565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:08.237010 32338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } attempt: 2
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265228, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265227, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
I20260809 08:47:08.242304 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.234856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.31.63.124:46297)
0809 08:47:08.234888 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.234890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.234897 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:08.234939 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.234939 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.234949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.235012 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.235217 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.235834 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.236647 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.237023 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.237095 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.239090 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.239092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.239092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.239099 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.240934 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.241387 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.241388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.241388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.241450 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.241711 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.241712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.241713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.241867 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.241928 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.242077 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.242207 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":17}
I20260809 08:47:08.242304 32568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.234802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56830)
0809 08:47:08.234836 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.234845 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:08.234891 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:08.235014 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:08.235014 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:08.235123 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:08.235152 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.235153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.235943 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.235943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.236566 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.236568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.237111 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.237163 (+ 52us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.237217 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.237706 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:08.237711 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:08.238994 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:08.241035 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.241036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.241036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.241265 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.241523 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.241524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.241525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.241633 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.241930 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:08.241931 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:08.241968 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 08:47:08.242054 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:47:08.242160 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 08:47:08.242872 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.233061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33582 (local address 127.31.63.126:33301)
0809 08:47:08.233101 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.233103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.233110 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:08.233131 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.233132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.233143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.233199 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.233845 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.234689 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.235629 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.235819 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.235870 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.238445 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.238447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.238447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.238456 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.240161 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.241524 (+ 1363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.241584 (+ 60us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.241585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.241645 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.241892 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.241893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.241894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.242044 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.242246 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.242664 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.242759 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:47:08.244824 32565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788: Bootstrap starting.
I20260809 08:47:08.245121 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.237300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56836)
0809 08:47:08.237349 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.237359 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:08.237398 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:08.237680 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:08.237681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:08.237853 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:08.237892 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.237893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.238811 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.238812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.239535 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.239538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.240050 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.240051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.240111 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.240388 (+ 277us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:08.240394 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:08.241649 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:08.244035 (+ 2386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.244036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.244037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.244214 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.244439 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.244440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.244440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.244501 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.244784 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:08.244785 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:08.244832 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 08:47:08.244924 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:47:08.245023 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 08:47:08.245425 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.237361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56836 (local address 127.31.63.124:46297)
0809 08:47:08.237518 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.237520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.237548 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:47:08.237566 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.237567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.237577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.237967 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.238020 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.238706 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.239647 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.239983 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.240050 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.241733 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.241734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.241734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.241740 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.243954 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.244304 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.244305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.244305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.244365 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.244574 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.244575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.244576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.244711 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.244791 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.245000 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.245073 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260809 08:47:08.245677 32565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:08.246011 32565 log.cc:824] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:08.247030 32338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42: Bootstrap starting.
I20260809 08:47:08.247215 32565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788: No bootstrap required, opened a new log
I20260809 08:47:08.247723 32338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:08.248018 32338 log.cc:824] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42: Log is configured to *not* fsync() on all Append() calls
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
I20260809 08:47:08.248828 32338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42: No bootstrap required, opened a new log
I20260809 08:47:08.249691 32565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } }
I20260809 08:47:08.249864 32565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0ffef8759b42a18517608fa7f87788, State: Initialized, Role: FOLLOWER
I20260809 08:47:08.249955 32565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } }
I20260809 08:47:08.250313 32565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:08.250829 32338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } }
I20260809 08:47:08.251029 32338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f0f322020614aceb561c34d5a05aa42, State: Initialized, Role: FOLLOWER
I20260809 08:47:08.251121 32338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } }
I20260809 08:47:08.251487 32338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:08.251448 32591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } } }
I20260809 08:47:08.251637 32591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:08.252405 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.243510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54490 (local address 127.31.63.125:40279)
0809 08:47:08.243553 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.243555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.243580 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:47:08.243608 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.243609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.243618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.243682 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.244267 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.244942 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.245928 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.246236 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.246287 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.249043 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.249045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.249046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.249055 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.250888 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.251327 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.251329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.251329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.251390 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.251630 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.251631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.251631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.251804 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.251963 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.252204 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.252293 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:47:08.252426 32602 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:08.252772 32602 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:08.253259 32579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } } }
I20260809 08:47:08.253369 32579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:47:08.253664 32622 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:08.253723 32622 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:08.252727 32456 master_runner.cc:186] Error getting master registration for 127.31.63.126:33301: OK, instance_id { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" instance_seqno: 1786265227891816 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:47:08.256106 32456 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:47:08.254390 32240 master_runner.cc:186] Error getting master registration for 127.31.63.125:40279: OK, instance_id { permanent_uuid: "048a4935d60e41779be0af14354852dc" instance_seqno: 1786265228052017 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:47:08.256891 32240 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:47:08.257726 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:08.258790 32297 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2f0f322020614aceb561c34d5a05aa42"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33582
I20260809 08:47:08.258912 32297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 08:47:08.258963 32297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:08.259938 32297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } }
I20260809 08:47:08.260238 32297 leader_election.cc:288] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [CANDIDATE]: Term 1 election: Requested vote from peers 048a4935d60e41779be0af14354852dc (127.31.63.125:40279), 3d0ffef8759b42a18517608fa7f87788 (127.31.63.124:46297)
I20260809 08:47:08.260563 32527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2f0f322020614aceb561c34d5a05aa42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3d0ffef8759b42a18517608fa7f87788"
I20260809 08:47:08.260684 32527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:08.261689 32527 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f0f322020614aceb561c34d5a05aa42 in term 1.
I20260809 08:47:08.261967 32261 leader_election.cc:302] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f0f322020614aceb561c34d5a05aa42, 3d0ffef8759b42a18517608fa7f87788; no voters:
I20260809 08:47:08.262104 32591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:08.262300 32591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [term 1 LEADER]: Becoming Leader. State: Replica: 2f0f322020614aceb561c34d5a05aa42, State: Running, Role: LEADER
I20260809 08:47:08.262398 32591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } }
I20260809 08:47:08.262877 32591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f0f322020614aceb561c34d5a05aa42. Latest consensus state: current_term: 1 leader_uuid: "2f0f322020614aceb561c34d5a05aa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } } }
I20260809 08:47:08.263022 32591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:08.273164 32634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:08.273407 32634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:08.275691 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.263643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.31.63.126:33301)
0809 08:47:08.263680 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.263682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.264993 (+ 1311us) server_negotiation.cc:407] Connection header received
0809 08:47:08.265087 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.265088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.265099 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.265127 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.265866 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.266584 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.267997 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.268313 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.268355 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.271435 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.271437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.271438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.271445 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.273556 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.274799 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.274800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.274801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.274867 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.275078 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.275079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.275079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.275209 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.275293 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.275510 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.275593 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:47:08.276758 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.263399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54496 (local address 127.31.63.125:40279)
0809 08:47:08.263446 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.263448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.264006 (+ 558us) server_negotiation.cc:407] Connection header received
0809 08:47:08.264066 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.264067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.264080 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.264113 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.264901 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.265708 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.266833 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.267170 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.267239 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.272772 (+ 5533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.272773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.272785 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.272794 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.274768 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.275365 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.275367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.275367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.275443 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.276030 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.276031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.276032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.276198 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.276274 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.276574 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.276663 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:47:08.279667 32527 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [term 1 FOLLOWER]: Refusing update from remote peer 2f0f322020614aceb561c34d5a05aa42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:08.280000 32625 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:08.280704 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.263874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56844 (local address 127.31.63.124:46297)
0809 08:47:08.263922 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.263924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.268740 (+ 4816us) server_negotiation.cc:407] Connection header received
0809 08:47:08.268844 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.268845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.268857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.268886 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.269660 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.270428 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.271437 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.271773 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.271813 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.276430 (+ 4617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.276432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.276432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.276442 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.278149 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.278547 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.278548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.278549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.278617 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.279229 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.279230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.279231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.279414 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.279526 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.280511 (+ 985us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.280609 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:47:08.281471 32579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f0f322020614aceb561c34d5a05aa42. Latest consensus state: current_term: 1 leader_uuid: "2f0f322020614aceb561c34d5a05aa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } } }
I20260809 08:47:08.281601 32579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:08.284053 32625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f0f322020614aceb561c34d5a05aa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } } }
I20260809 08:47:08.284143 32625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:08.284547 32579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f0f322020614aceb561c34d5a05aa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } } }
I20260809 08:47:08.284639 32579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:08.284740 32638 mvcc.cc:205] Tried to move back new op lower bound from 7316542375030026240 to 7316542374963630080. Current Snapshot: MvccSnapshot[applied={T|T < 7316542375030026240}]
I20260809 08:47:08.284876 32639 mvcc.cc:205] Tried to move back new op lower bound from 7316542375030026240 to 7316542374963630080. Current Snapshot: MvccSnapshot[applied={T|T < 7316542375030026240}]
I20260809 08:47:08.285272 32634 catalog_manager.cc:1347] Generated new cluster ID: e9ed271d96f2427c894e0157c1f7d611
I20260809 08:47:08.285329 32634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:08.292099 32449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } attempt: 3
Aug 09 08:47:08 dist-test-slave-gkw7 krb5kdc[32224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265228, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
I20260809 08:47:08.299403 32634 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:08.300148 32634 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:08.300917 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.292368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:46297 (local address 127.0.0.1:56848)
0809 08:47:08.292417 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:08.292429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:08.292467 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:08.292637 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:08.292639 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:08.292844 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:08.292902 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.292904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.293872 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.293874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.294695 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:08.294698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.295183 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.295184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:08.295263 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.295928 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:08.295933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:08.297613 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:08.299675 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.299676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.299677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.299918 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.300139 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:08.300140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:08.300141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:08.300206 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:08.300567 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:08.300568 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:08.300618 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 08:47:08.300687 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:47:08.300797 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:47:08.301709 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:08.292419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.31.63.124:46297)
0809 08:47:08.292457 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.292459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.292469 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:08.292548 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.292550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.292562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.292614 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.292986 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.293774 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.294793 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:08.295090 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:08.295152 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:08.297712 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:08.297714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:08.297714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:08.297722 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:08.299582 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.299979 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.299980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.299980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.300056 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:08.300306 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:08.300306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:08.300307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:08.300463 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:08.300527 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:08.301484 (+ 957us) server_negotiation.cc:299] Negotiation successful
0809 08:47:08.301607 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:47:08.303328 32449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc: Bootstrap starting.
I20260809 08:47:08.304189 32449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:08.304533 32449 log.cc:824] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:08.310501 32449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc: No bootstrap required, opened a new log
I20260809 08:47:08.311101 32634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42: Generated new TSK 0
I20260809 08:47:08.311303 32634 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:08.312858 32449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } }
I20260809 08:47:08.313019 32449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048a4935d60e41779be0af14354852dc, State: Initialized, Role: FOLLOWER
I20260809 08:47:08.313112 32449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } }
I20260809 08:47:08.313506 32449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:08.313625 32408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:08.314592 32408 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [term 1 FOLLOWER]: Refusing update from remote peer 2f0f322020614aceb561c34d5a05aa42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:08.314893 32591 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:08.315008 32409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2f0f322020614aceb561c34d5a05aa42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "048a4935d60e41779be0af14354852dc"
I20260809 08:47:08.316973 32670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f0f322020614aceb561c34d5a05aa42. Latest consensus state: current_term: 1 leader_uuid: "2f0f322020614aceb561c34d5a05aa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } } }
I20260809 08:47:08.317094 32670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:08.317178 32670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f0f322020614aceb561c34d5a05aa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } } }
I20260809 08:47:08.317260 32670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:08.318048 32409 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2f0f322020614aceb561c34d5a05aa42 for term 1 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260809 08:47:08.318912 32667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f0f322020614aceb561c34d5a05aa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } } }
I20260809 08:47:08.319031 32667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:47:08.323913 32666 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:08.323978 32666 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:08.324591 32668 mvcc.cc:205] Tried to move back new op lower bound from 7316542375156527104 to 7316542374963630080. Current Snapshot: MvccSnapshot[applied={T|T < 7316542375030026240}]
W20260809 08:47:08.337499 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.336091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54512 (local address 127.31.63.125:40279)
0809 08:47:08.336134 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.336137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.336803 (+ 666us) server_negotiation.cc:407] Connection header received
0809 08:47:08.336911 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.336912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.336924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.336956 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.337422 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54512: BlockingRecv error: recv got EOF from 127.0.0.1:54512 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:47:08.338413 32344 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.336511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.31.63.126:33301)
0809 08:47:08.336553 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.336555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.337871 (+ 1316us) server_negotiation.cc:407] Connection header received
0809 08:47:08.337972 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.337973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.337984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.338014 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.338348 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33596: BlockingRecv error: recv got EOF from 127.0.0.1:33596 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 08:47:08.339620 32576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.338859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.31.63.124:46297)
0809 08:47:08.338902 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.338904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.339044 (+ 140us) server_negotiation.cc:407] Connection header received
0809 08:47:08.339169 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.339171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.339182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.339218 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.339554 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56864: BlockingRecv error: recv got EOF from 127.0.0.1:56864 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:47:08.346612 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.345832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54528 (local address 127.31.63.125:40279)
0809 08:47:08.345870 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.345872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.346053 (+ 181us) server_negotiation.cc:407] Connection header received
0809 08:47:08.346179 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.346181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.346200 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.346231 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.346563 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54528: BlockingRecv error: recv got EOF from 127.0.0.1:54528 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:47:08.349778 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.348950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54534 (local address 127.31.63.125:40279)
0809 08:47:08.348984 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.348986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.349254 (+ 268us) server_negotiation.cc:407] Connection header received
0809 08:47:08.349359 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.349360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.349370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.349397 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.349733 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54534: BlockingRecv error: recv got EOF from 127.0.0.1:54534 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:47:08.353749 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.352910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54536 (local address 127.31.63.125:40279)
0809 08:47:08.352942 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.352944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.353250 (+ 306us) server_negotiation.cc:407] Connection header received
0809 08:47:08.353344 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.353345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.353355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.353383 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.353706 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54536: BlockingRecv error: recv got EOF from 127.0.0.1:54536 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:47:08.359614 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.358827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54548 (local address 127.31.63.125:40279)
0809 08:47:08.358863 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.358865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.359105 (+ 240us) server_negotiation.cc:407] Connection header received
0809 08:47:08.359218 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.359219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.359229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.359257 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.359570 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54548: BlockingRecv error: recv got EOF from 127.0.0.1:54548 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:47:08.369755 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.368858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54556 (local address 127.31.63.125:40279)
0809 08:47:08.368895 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.368896 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.369170 (+ 274us) server_negotiation.cc:407] Connection header received
0809 08:47:08.369273 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.369274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.369285 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.369314 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.369693 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54556: BlockingRecv error: recv got EOF from 127.0.0.1:54556 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:47:08.388103 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.387142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54568 (local address 127.31.63.125:40279)
0809 08:47:08.387222 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.387224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.387512 (+ 288us) server_negotiation.cc:407] Connection header received
0809 08:47:08.387617 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.387618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.387630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.387660 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.388044 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54568: BlockingRecv error: recv got EOF from 127.0.0.1:54568 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 08:47:08.422783 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.421722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54572 (local address 127.31.63.125:40279)
0809 08:47:08.421772 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.421774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.422114 (+ 340us) server_negotiation.cc:407] Connection header received
0809 08:47:08.422247 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.422249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.422262 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.422296 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.422718 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54572: BlockingRecv error: recv got EOF from 127.0.0.1:54572 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 08:47:08.489176 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.488382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54588 (local address 127.31.63.125:40279)
0809 08:47:08.488447 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.488450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.488602 (+ 152us) server_negotiation.cc:407] Connection header received
0809 08:47:08.488637 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.488638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.488651 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.488707 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.489106 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54588: BlockingRecv error: recv got EOF from 127.0.0.1:54588 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 08:47:08.619321 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.618600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54594 (local address 127.31.63.125:40279)
0809 08:47:08.618698 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.618701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.618772 (+ 71us) server_negotiation.cc:407] Connection header received
0809 08:47:08.618823 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.618824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.618835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.618889 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.619246 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54594: BlockingRecv error: recv got EOF from 127.0.0.1:54594 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 08:47:08.838708 32760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f0f322020614aceb561c34d5a05aa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0f322020614aceb561c34d5a05aa42" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 33301 } } peers { permanent_uuid: "048a4935d60e41779be0af14354852dc" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 40279 } } peers { permanent_uuid: "3d0ffef8759b42a18517608fa7f87788" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 46297 } } }
I20260809 08:47:08.838892 32760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0f322020614aceb561c34d5a05aa42 [sys.catalog]: This master's current role is: LEADER
W20260809 08:47:08.877364 32454 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:08.876626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54596 (local address 127.31.63.125:40279)
0809 08:47:08.876699 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:08.876701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:08.876711 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:08.876790 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:08.876791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:08.876810 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:08.876878 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:08.877314 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54596: BlockingRecv error: recv got EOF from 127.0.0.1:54596 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 08:47:09.253423 32602 catalog_manager.cc:1259] Loaded cluster ID: e9ed271d96f2427c894e0157c1f7d611
I20260809 08:47:09.253501 32602 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788: loading cluster ID for follower catalog manager: success
I20260809 08:47:09.254380 32602 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788: acquiring CA information for follower catalog manager: success
I20260809 08:47:09.254709 32602 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d0ffef8759b42a18517608fa7f87788: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:47:09.324455 32666 catalog_manager.cc:1259] Loaded cluster ID: e9ed271d96f2427c894e0157c1f7d611
I20260809 08:47:09.324573 32666 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc: loading cluster ID for follower catalog manager: success
I20260809 08:47:09.325430 32666 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc: acquiring CA information for follower catalog manager: success
I20260809 08:47:09.325810 32666 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 048a4935d60e41779be0af14354852dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:47:09.393904 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:09.390966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.31.63.125:40279)
0809 08:47:09.391093 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:09.391096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:09.391223 (+ 127us) server_negotiation.cc:407] Connection header received
0809 08:47:09.391323 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:09.391324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:09.391387 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:09.391477 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:09.391652 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:09.392170 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:09.392992 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:09.393200 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:09.393294 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:09.393489 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:09.393628 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:09.393716 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:09.393777 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 08:47:09.393802 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260809 08:47:09.398517 319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:09.395941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56880 (local address 127.31.63.124:46297)
0809 08:47:09.396072 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:09.396075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:09.396088 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:47:09.396165 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:09.396166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:09.396221 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:09.396298 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:09.396422 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:09.396957 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:09.397688 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:09.397964 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:09.398023 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:09.398158 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:09.398266 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:09.398328 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:09.398364 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 08:47:09.398385 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260809 08:47:09.403095 319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:09.400408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56886 (local address 127.31.63.124:46297)
0809 08:47:09.400454 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:09.400456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:09.400464 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:09.400487 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:09.400487 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:09.400505 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:09.400591 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:09.400760 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:09.401338 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:09.402321 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:09.402618 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:09.402681 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:09.402839 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:09.402870 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:09.402923 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:09.403004 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 08:47:09.403027 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:47:09.403569 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:09.400296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54622 (local address 127.31.63.125:40279)
0809 08:47:09.400346 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:09.400348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:09.400548 (+ 200us) server_negotiation.cc:407] Connection header received
0809 08:47:09.400607 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:09.400609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:09.400620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:09.400674 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:09.401037 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:09.401775 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:09.402670 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:09.403012 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:09.403076 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:09.403287 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:09.403323 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:09.403362 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:09.403474 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 08:47:09.403500 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:47:09.404603 330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:09.400175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.31.63.126:33301)
0809 08:47:09.400310 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:09.400312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:09.400711 (+ 399us) server_negotiation.cc:407] Connection header received
0809 08:47:09.400826 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:09.400827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:09.400875 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:09.400973 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:09.401134 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:09.401814 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:09.403683 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:09.403980 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:09.404060 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:09.404268 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:09.404358 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:09.404427 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:09.404490 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 08:47:09.404515 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260809 08:47:09.406262 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 32240
I20260809 08:47:09.416790 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 32349
I20260809 08:47:09.429080 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 32456
2026-08-09T08:47:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3036 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:09 dist-test-slave-gkw7 krb5kdc[336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:09 dist-test-slave-gkw7 krb5kdc[336](info): set up 2 sockets
Aug 09 08:47:09 dist-test-slave-gkw7 krb5kdc[336](info): commencing operation
krb5kdc: starting...
W20260809 08:47:10.676060 31997 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/test-user.keytab.
Aug 09 08:47:10 dist-test-slave-gkw7 krb5kdc[336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
Entry for principal kudu/127.31.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab.
I20260809 08:47:10.804883 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:35429
--webserver_interface=127.31.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:46467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:35429,127.31.63.125:33351,127.31.63.124:37795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:10.888931 353 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:47:10.889211 353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:10.889245 353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:10.889289 353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:10.890467 353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:10.890540 353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:10.890568 353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:10.890589 353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:10.890614 353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:10.892169 353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:35429,127.31.63.125:33351,127.31.63.124:37795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.126.keytab
--principal=kudu/127.31.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.126:35429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:10.892468 353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:10.892709 353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:10.897897 366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:10.897903 364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:10.897907 363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:10.898233 353 server_base.cc:1034] running on GCE node
I20260809 08:47:10.898532 353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:10.898870 353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:10.900068 353 hybrid_clock.cc:648] HybridClock initialized: now 1786265230900052 us; error 31 us; skew 500 ppm
Aug 09 08:47:10 dist-test-slave-gkw7 krb5kdc[336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265230, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:10.907542 353 init.cc:377] Logged in from keytab as kudu/127.31.63.126@KRBTEST.COM (short username kudu)
I20260809 08:47:10.908064 353 webserver.cc:460] Webserver started at http://127.31.63.126:37593/ using document root <none> and password file <none>
I20260809 08:47:10.908293 353 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:10.908339 353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:10.908440 353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:10.909581 353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/data/instance:
uuid: "645ae096ae834f838750235f4b8ea021"
format_stamp: "Formatted at 2026-08-09 08:47:10 on dist-test-slave-gkw7"
server_key: "c4f0899409d5a259546d87dd6e54bfa6"
server_key_iv: "5f3bdbbccf5f39d33852ee407b91f4ce"
server_key_version: "encryptionkey@0"
I20260809 08:47:10.909971 353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/wal/instance:
uuid: "645ae096ae834f838750235f4b8ea021"
format_stamp: "Formatted at 2026-08-09 08:47:10 on dist-test-slave-gkw7"
server_key: "c4f0899409d5a259546d87dd6e54bfa6"
server_key_iv: "5f3bdbbccf5f39d33852ee407b91f4ce"
server_key_version: "encryptionkey@0"
I20260809 08:47:10.911443 353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:10.912226 377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:10.912431 353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:10.912518 353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/wal
uuid: "645ae096ae834f838750235f4b8ea021"
format_stamp: "Formatted at 2026-08-09 08:47:10 on dist-test-slave-gkw7"
server_key: "c4f0899409d5a259546d87dd6e54bfa6"
server_key_iv: "5f3bdbbccf5f39d33852ee407b91f4ce"
server_key_version: "encryptionkey@0"
I20260809 08:47:10.912573 353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:10.927037 353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:10.929033 353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:10.929149 353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:10.933386 353 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.126:35429
I20260809 08:47:10.933522 455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.126:35429 every 8 connection(s)
I20260809 08:47:10.933864 353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:10.934978 456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:10.936618 456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:10.940893 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 353
I20260809 08:47:10.940975 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-0/wal/instance
I20260809 08:47:10.941203 31997 external_mini_cluster.cc:1428] Setting key eedaa3be23ff88737e47adf7447e958c
Aug 09 08:47:10 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265230, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:47:10.948042 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:10.937347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:35429 (local address 127.0.0.1:49404)
0809 08:47:10.937726 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:10.937774 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:10.938291 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:10.938611 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:10.938614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:10.938902 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:10.939059 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:10.939064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:10.940079 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:10.940080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:10.940760 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:10.940762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:10.941404 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:10.941405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:10.941474 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:10.942110 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:10.942118 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:10.943733 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:10.946022 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:10.946029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:10.946031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:10.946269 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:10.946619 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:10.946620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:10.946621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:10.946721 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:10.947376 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:10.947381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:10.947469 (+ 88us) client_negotiation.cc:769] Sending connection context
0809 08:47:10.947583 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 08:47:10.947848 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":105,"thread_start_us":136,"threads_started":1}
I20260809 08:47:10.948056 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:10.937434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49404 (local address 127.31.63.126:35429)
0809 08:47:10.937638 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:10.937655 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:10.937873 (+ 218us) server_negotiation.cc:407] Connection header received
0809 08:47:10.938456 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:10.938459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:10.938485 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:10.938590 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:10.939164 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:10.939967 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:10.940859 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:10.941300 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:10.941422 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:10.943826 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:10.943835 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:10.943835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:10.943845 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:10.945919 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:10.946427 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:10.946431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:10.946431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:10.946524 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:10.946793 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:10.946795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:10.946795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:10.947104 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:10.947409 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:10.947624 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 08:47:10.947779 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
I20260809 08:47:10.948873 456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:10.949272 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:10.949139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:33351 (local address 127.0.0.1:59596)
0809 08:47:10.949192 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:10.949239 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:33351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 08:47:10.949396 381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.63.125:33351: connect: Connection refused (error 111)
I20260809 08:47:10.949887 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:10.949792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:33351 (local address 127.0.0.1:59604)
0809 08:47:10.949840 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:10.949865 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:33351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 08:47:10.950078 456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.125:33351: Network error: Client connection negotiation failed: client connection to 127.31.63.125:33351: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.63.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab.
Entry for principal kudu/127.31.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab.
I20260809 08:47:10.971982 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.125:33351
--webserver_interface=127.31.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab
--principal=kudu/127.31.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:46467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:35429,127.31.63.125:33351,127.31.63.124:37795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:47:10.999307 456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } attempt: 1
I20260809 08:47:10.999869 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:10.999712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:33351 (local address 127.0.0.1:59612)
0809 08:47:10.999781 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:10.999832 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:33351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 08:47:11.000278 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.000182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:33351 (local address 127.0.0.1:59624)
0809 08:47:11.000229 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.000258 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:33351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 08:47:11.000414 456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.125:33351: Network error: Client connection negotiation failed: client connection to 127.31.63.125:33351: connect: Connection refused (error 111)
W20260809 08:47:11.058408 470 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:47:11.058629 470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:11.058655 470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:11.058676 470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:11.059892 470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:11.059937 470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:11.059952 470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:11.059964 470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:11.059978 470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:11.061583 470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:35429,127.31.63.125:33351,127.31.63.124:37795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.125.keytab
--principal=kudu/127.31.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.125:33351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:11.061906 470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:11.062178 470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:11.067121 482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:11.067121 479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:11.067158 478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:11.067466 470 server_base.cc:1034] running on GCE node
I20260809 08:47:11.067688 470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:11.067946 470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:11.068588 456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } attempt: 2
I20260809 08:47:11.068976 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.068838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:33351 (local address 127.0.0.1:59626)
0809 08:47:11.068909 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.068946 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:33351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 08:47:11.069105 470 hybrid_clock.cc:648] HybridClock initialized: now 1786265231069084 us; error 32 us; skew 500 ppm
I20260809 08:47:11.069370 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.069280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:33351 (local address 127.0.0.1:59636)
0809 08:47:11.069317 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.069342 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.125:33351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 08:47:11.069468 456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.125:33351: Network error: Client connection negotiation failed: client connection to 127.31.63.125:33351: connect: Connection refused (error 111)
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:11.072021 470 init.cc:377] Logged in from keytab as kudu/127.31.63.125@KRBTEST.COM (short username kudu)
I20260809 08:47:11.072486 470 webserver.cc:460] Webserver started at http://127.31.63.125:37959/ using document root <none> and password file <none>
I20260809 08:47:11.072757 470 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:11.072831 470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:11.072981 470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:11.074162 470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/data/instance:
uuid: "283899fb6d894182b05b622f15f07d55"
format_stamp: "Formatted at 2026-08-09 08:47:11 on dist-test-slave-gkw7"
server_key: "d8c71d5715858af81243edd9ad3552a2"
server_key_iv: "a58e47f31d2941a282c67acaecae86c1"
server_key_version: "encryptionkey@0"
I20260809 08:47:11.074534 470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/wal/instance:
uuid: "283899fb6d894182b05b622f15f07d55"
format_stamp: "Formatted at 2026-08-09 08:47:11 on dist-test-slave-gkw7"
server_key: "d8c71d5715858af81243edd9ad3552a2"
server_key_iv: "a58e47f31d2941a282c67acaecae86c1"
server_key_version: "encryptionkey@0"
I20260809 08:47:11.075953 470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 08:47:11.076785 492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:11.076992 470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:11.077102 470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/wal
uuid: "283899fb6d894182b05b622f15f07d55"
format_stamp: "Formatted at 2026-08-09 08:47:11 on dist-test-slave-gkw7"
server_key: "d8c71d5715858af81243edd9ad3552a2"
server_key_iv: "a58e47f31d2941a282c67acaecae86c1"
server_key_version: "encryptionkey@0"
I20260809 08:47:11.077198 470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:11.113754 470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:11.115859 470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:11.116035 470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:11.120555 470 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.125:33351
I20260809 08:47:11.120612 569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.125:33351 every 8 connection(s)
I20260809 08:47:11.121016 470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/data/info.pb
I20260809 08:47:11.122133 570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:11.123677 570 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:11.128332 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 470
I20260809 08:47:11.128443 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-1/wal/instance
I20260809 08:47:11.128654 31997 external_mini_cluster.cc:1428] Setting key f2ed377d3fafa0d23869c7f3871f7888
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:47:11.133431 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.124391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.31.63.126:35429)
0809 08:47:11.124455 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.124459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.124672 (+ 213us) server_negotiation.cc:407] Connection header received
0809 08:47:11.125154 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.125155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.125168 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.125228 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.126018 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.126512 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.127475 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.127677 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.127728 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.130331 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.130333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.130333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.130341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.131859 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.132403 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.132404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.132404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.132465 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.132772 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.132773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.132773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.132885 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.132936 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.133283 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.133364 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:47:11.133812 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.124329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:35429 (local address 127.0.0.1:49418)
0809 08:47:11.124531 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.124578 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:11.125070 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:11.125314 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:11.125317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:11.125660 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:11.125927 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.125934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.126590 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.126591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.127398 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.127400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.127765 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.127766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.127871 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.128454 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:11.128466 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:11.130214 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:11.131966 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.131975 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.131978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.132318 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.132540 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.132541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.132541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.132696 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.132952 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:11.132959 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:11.133133 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 08:47:11.133261 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 08:47:11.133449 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
I20260809 08:47:11.134531 570 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.63.124@KRBTEST.COM; defaulting to no policy
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.63.124@KRBTEST.COM" created.
I20260809 08:47:11.142266 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.134828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:33351 (local address 127.0.0.1:59650)
0809 08:47:11.134891 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.134905 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:11.134954 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:11.135174 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:11.135176 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:11.135348 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:11.135423 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.135425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.136267 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.136268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.137108 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.137111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.137658 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.137660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.137739 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.138118 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:11.138124 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:11.139456 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:11.141116 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.141117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.141117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.141301 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.141497 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.141497 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.141497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.141597 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.141980 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:11.141981 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:11.142021 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 08:47:11.142072 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 08:47:11.142152 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 08:47:11.142369 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.134865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59650 (local address 127.31.63.125:33351)
0809 08:47:11.135029 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.135034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.135049 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:47:11.135068 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.135069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.135080 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.135142 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.135502 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.136177 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.137206 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.137507 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.137662 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.139532 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.139539 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.139539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.139546 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.141018 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.141376 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.141379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.141379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.141436 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.141667 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.141668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.141668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.141903 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.141988 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.142111 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.142226 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260809 08:47:11.143000 570 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:11.143378 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.143260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41864)
0809 08:47:11.143312 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.143358 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:47:11.143530 498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
I20260809 08:47:11.143963 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.143874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41874)
0809 08:47:11.143921 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.143946 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 08:47:11.144071 570 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:37795: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab.
Entry for principal kudu/127.31.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab.
I20260809 08:47:11.159823 31997 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskslFlM2/build/release/bin/kudu
/tmp/dist-test-taskslFlM2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.124:37795
--webserver_interface=127.31.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab
--principal=kudu/127.31.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.63.84:46467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.63.126:35429,127.31.63.125:33351,127.31.63.124:37795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:47:11.160605 456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } attempt: 3
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265230, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
I20260809 08:47:11.168684 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.161035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59662 (local address 127.31.63.125:33351)
0809 08:47:11.161109 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.161111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.161121 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:11.161171 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.161172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.161184 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.161237 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.161691 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.162532 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.163261 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.163631 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.163711 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.166078 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.166079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.166079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.166086 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.167505 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.167935 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.167936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.167936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.167987 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.168209 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.168210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.168210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.168312 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.168361 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.168552 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.168614 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 08:47:11.168790 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.160964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:33351 (local address 127.0.0.1:59662)
0809 08:47:11.161027 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.161043 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:11.161120 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:11.161271 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:11.161272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:11.161472 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:11.161592 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.161595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.162630 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.162631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.163170 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.163171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.163714 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.163715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.163776 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.164307 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:11.164312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:11.165984 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:11.167595 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.167596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.167597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.167851 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.168055 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.168056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.168056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.168133 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.168381 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:11.168382 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:11.168442 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 08:47:11.168530 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 08:47:11.168644 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 08:47:11.169292 456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:11.169694 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.169579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41876)
0809 08:47:11.169621 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.169658 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 08:47:11.170049 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.169927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41892)
0809 08:47:11.169989 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.170018 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 08:47:11.170163 456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:37795: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
I20260809 08:47:11.193367 570 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } attempt: 1
I20260809 08:47:11.193971 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.193816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41908)
0809 08:47:11.193883 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.193931 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 08:47:11.194361 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.194258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41924)
0809 08:47:11.194309 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.194339 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:47:11.194464 570 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:37795: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
I20260809 08:47:11.201318 456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } attempt: 1
I20260809 08:47:11.201805 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.201654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41932)
0809 08:47:11.201719 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.201760 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:47:11.202186 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.202080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41942)
0809 08:47:11.202131 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.202167 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:47:11.202286 456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:37795: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
W20260809 08:47:11.247627 580 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:47:11.247850 580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:11.247869 580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:11.247887 580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:11.249054 580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:11.249099 580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:11.249112 580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:11.249125 580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:11.249189 580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:11.250749 580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.63.84:46467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.63.126:35429,127.31.63.125:33351,127.31.63.124:37795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/krb5kdc/kudu_127.31.63.124.keytab
--principal=kudu/127.31.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.63.124:37795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:11.251019 580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:11.251272 580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:11.256208 586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:11.256263 589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:11.256309 585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:11.256677 580 server_base.cc:1034] running on GCE node
I20260809 08:47:11.256927 580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:11.257237 580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:11.258421 580 hybrid_clock.cc:648] HybridClock initialized: now 1786265231258416 us; error 37 us; skew 500 ppm
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:11.261021 580 init.cc:377] Logged in from keytab as kudu/127.31.63.124@KRBTEST.COM (short username kudu)
I20260809 08:47:11.261515 580 webserver.cc:460] Webserver started at http://127.31.63.124:45371/ using document root <none> and password file <none>
I20260809 08:47:11.261830 580 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:11.261878 580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:11.261974 580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:11.262610 570 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } attempt: 2
I20260809 08:47:11.263043 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.262900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41952)
0809 08:47:11.262969 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.263009 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 08:47:11.263437 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.263326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41966)
0809 08:47:11.263380 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.263413 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 08:47:11.263295 580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/data/instance:
uuid: "21cbce6ab5274ba8aa1d49b98cc9002c"
format_stamp: "Formatted at 2026-08-09 08:47:11 on dist-test-slave-gkw7"
server_key: "f2810ed96a69b8e8f3764682f1fad3b9"
server_key_iv: "55004703674e8092364b57b1764737ea"
server_key_version: "encryptionkey@0"
W20260809 08:47:11.263540 570 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:37795: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
I20260809 08:47:11.263743 580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/wal/instance:
uuid: "21cbce6ab5274ba8aa1d49b98cc9002c"
format_stamp: "Formatted at 2026-08-09 08:47:11 on dist-test-slave-gkw7"
server_key: "f2810ed96a69b8e8f3764682f1fad3b9"
server_key_iv: "55004703674e8092364b57b1764737ea"
server_key_version: "encryptionkey@0"
I20260809 08:47:11.265115 580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:47:11.265974 602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:11.266151 580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:47:11.266256 580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/data,/tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/wal
uuid: "21cbce6ab5274ba8aa1d49b98cc9002c"
format_stamp: "Formatted at 2026-08-09 08:47:11 on dist-test-slave-gkw7"
server_key: "f2810ed96a69b8e8f3764682f1fad3b9"
server_key_iv: "55004703674e8092364b57b1764737ea"
server_key_version: "encryptionkey@0"
I20260809 08:47:11.266342 580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:11.277582 456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } attempt: 2
I20260809 08:47:11.278053 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.277904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41976)
0809 08:47:11.277976 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.278018 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 08:47:11.278985 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.278908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:41986)
0809 08:47:11.278945 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.278963 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 08:47:11.279054 456 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.63.124:37795: Network error: Client connection negotiation failed: client connection to 127.31.63.124:37795: connect: Connection refused (error 111)
I20260809 08:47:11.287762 580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:11.290215 580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:11.290334 580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:11.294668 580 rpc_server.cc:307] RPC server started. Bound to: 127.31.63.124:37795
I20260809 08:47:11.294699 678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.63.124:37795 every 8 connection(s)
I20260809 08:47:11.295070 580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/data/info.pb
I20260809 08:47:11.296073 680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:11.296357 31997 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskslFlM2/build/release/bin/kudu as pid 580
I20260809 08:47:11.296471 31997 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskslFlM2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786264918773806-31997-0/minicluster-data/master-2/wal/instance
I20260809 08:47:11.296695 31997 external_mini_cluster.cc:1428] Setting key d8ab24f3404392c2d95c6ca8dbd0f993
I20260809 08:47:11.298782 680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:47:11.307641 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.299553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.31.63.126:35429)
0809 08:47:11.299619 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.299622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.299754 (+ 132us) server_negotiation.cc:407] Connection header received
0809 08:47:11.299820 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.299821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.299832 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.299889 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.300329 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.300776 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.301644 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.301914 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.301986 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.304848 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.304850 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.304850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.304858 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.306367 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.306718 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.306719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.306719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.306790 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.307023 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.307024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.307024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.307145 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.307214 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.307437 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.307518 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:47:11.307770 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.297261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41998 (local address 127.31.63.124:37795)
0809 08:47:11.297436 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.297446 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.297481 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:47:11.297974 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.297981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.298029 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.298141 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.298987 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.300184 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.301032 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.301435 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.301513 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.304285 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.304293 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.304293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.304299 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.306300 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.306664 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.306670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.306670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.306753 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.306964 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.306965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.306965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.307174 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.307333 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.307405 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.307553 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260809 08:47:11.307899 685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.299478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.126:35429 (local address 127.0.0.1:49422)
0809 08:47:11.299645 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.299683 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:11.299755 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:11.299902 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:11.299903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:11.300133 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:11.300215 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.300246 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.300868 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.300869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.301570 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.301573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.302039 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.302040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.302119 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.302519 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:11.302531 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:11.304761 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:11.306435 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.306440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.306443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.306632 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.306861 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.306862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.306863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.306949 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.307218 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:11.307225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:11.307305 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 08:47:11.307405 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 08:47:11.307713 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":165,"thread_start_us":116,"threads_started":1}
I20260809 08:47:11.308712 680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.126@KRBTEST.COM
I20260809 08:47:11.315618 685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.308960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.125:33351 (local address 127.0.0.1:59676)
0809 08:47:11.309087 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.309096 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:11.309151 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:11.309281 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:11.309281 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:11.309415 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:11.309456 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.309457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.310119 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.310120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.310595 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.310596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.310972 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.310973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.311030 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.311462 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:11.311466 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:11.312756 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:11.314523 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.314524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.314525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.314750 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.314977 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.314978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.314978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.315039 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.315301 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:11.315302 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:11.315345 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 08:47:11.315432 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 08:47:11.315526 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260809 08:47:11.315618 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.309014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.31.63.125:33351)
0809 08:47:11.309060 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.309062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.309130 (+ 68us) server_negotiation.cc:407] Connection header received
0809 08:47:11.309207 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.309208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.309218 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.309269 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.309551 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.310028 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.310682 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.310900 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.310975 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.312842 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.312843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.312844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.312851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.314441 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.314835 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.314837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.314837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.314899 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.315121 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.315122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.315122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.315234 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.315309 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.315458 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.315527 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:47:11.315982 680 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:11.316366 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.308901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49438 (local address 127.31.63.126:35429)
0809 08:47:11.308940 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.308941 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.308950 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:11.309040 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.309041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.309053 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.309115 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.309570 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.310027 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.310696 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.310879 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.310953 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.313495 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.313496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.313497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.313503 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.314977 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.315364 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.315365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.315365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.315422 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.315625 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.315626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.315626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.315776 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.315954 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.316167 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.316242 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.124@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.63.125@KRBTEST.COM
I20260809 08:47:11.322767 685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.316274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:42004)
0809 08:47:11.316305 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.316314 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:11.316353 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:11.316501 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:11.316501 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:11.316651 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:11.316691 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.316692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.317639 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.317640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.318097 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.318098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.318495 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.318496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.318543 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.318932 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:11.318935 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:11.320008 (+ 1073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:11.321621 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.321622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.321623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.321869 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.322047 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.322048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.322048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.322115 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.322401 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:11.322401 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:11.322436 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 08:47:11.322491 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:47:11.322602 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 08:47:11.322845 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.316306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42004 (local address 127.31.63.124:37795)
0809 08:47:11.316338 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.316339 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.316347 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:11.316408 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.316409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.316417 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.316468 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.316772 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.317513 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.318187 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.318426 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.318498 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.320091 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.320093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.320093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.320100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.321499 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.321950 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.321951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.321952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.322001 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.322177 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.322178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.322178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.322324 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.322399 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.322534 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.322648 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":18}
I20260809 08:47:11.324213 680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c: Bootstrap starting.
I20260809 08:47:11.324631 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.317054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59690 (local address 127.31.63.125:33351)
0809 08:47:11.317086 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.317088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.317126 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:47:11.317151 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.317151 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.317158 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.317195 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.317789 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.318222 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.319054 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.319283 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.319356 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.321753 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.321755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.321755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.321762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.323543 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.323871 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.323872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.323872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.323926 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.324124 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.324125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.324126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.324236 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.324411 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.324446 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.324514 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:47:11.325278 680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:11.325661 680 log.cc:824] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:11.326462 31997 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:11.326674 680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c: No bootstrap required, opened a new log
I20260809 08:47:11.327387 417 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "645ae096ae834f838750235f4b8ea021"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49438
I20260809 08:47:11.328613 680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } }
I20260809 08:47:11.328766 680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21cbce6ab5274ba8aa1d49b98cc9002c, State: Initialized, Role: FOLLOWER
I20260809 08:47:11.328845 680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } }
I20260809 08:47:11.329172 680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:11.329155 687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } } }
I20260809 08:47:11.329279 687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:47:11.332762 703 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:11.332820 703 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:11.332967 580 master_runner.cc:186] Error getting master registration for 127.31.63.126:35429: OK, instance_id { permanent_uuid: "645ae096ae834f838750235f4b8ea021" instance_seqno: 1786265230932152 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:47:11.335289 580 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:47:11.354664 570 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } attempt: 3
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265231, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.125@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
I20260809 08:47:11.363864 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.355099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:42016)
0809 08:47:11.355165 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.355174 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:11.355211 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:11.355378 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:11.355379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:11.355535 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:11.355585 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.355587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.356513 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.356514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.358223 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.358226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.358727 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.358728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.358794 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.359322 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:11.359327 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:11.360777 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:11.362718 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.362719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.362719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.362889 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.363116 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.363117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.363117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.363185 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.363532 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:11.363533 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:11.363583 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 08:47:11.363650 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:47:11.363754 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 08:47:11.363873 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.355151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.31.63.124:37795)
0809 08:47:11.355205 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.355207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.355221 (+ 14us) server_negotiation.cc:407] Connection header received
0809 08:47:11.355284 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.355284 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.355295 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.355351 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.355640 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.356419 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.358325 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.358642 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.358722 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.360879 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.360881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.360881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.360889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.362629 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.362963 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.362965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.362965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.363030 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.363254 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.363255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.363255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.363462 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.363536 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.363692 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.363776 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:47:11.365432 570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55: Bootstrap starting.
I20260809 08:47:11.366122 570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:11.366417 570 log.cc:824] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:11.367229 570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55: No bootstrap required, opened a new log
I20260809 08:47:11.368850 570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } }
I20260809 08:47:11.368963 570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283899fb6d894182b05b622f15f07d55, State: Initialized, Role: FOLLOWER
I20260809 08:47:11.369107 570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } }
I20260809 08:47:11.369423 709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } } }
I20260809 08:47:11.369504 570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:11.369520 709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:47:11.371369 723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:11.371424 723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:11.371709 470 master_runner.cc:186] Error getting master registration for 127.31.63.126:35429: OK, instance_id { permanent_uuid: "645ae096ae834f838750235f4b8ea021" instance_seqno: 1786265230932152 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:47:11.374553 470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:47:11.378178 456 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } attempt: 3
Aug 09 08:47:11 dist-test-slave-gkw7 krb5kdc[336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265230, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.63.126@KRBTEST.COM for kudu/127.31.63.124@KRBTEST.COM
I20260809 08:47:11.386788 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.378517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.63.124:37795 (local address 127.0.0.1:42032)
0809 08:47:11.378578 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:11.378590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:11.378631 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:11.378867 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:11.378869 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:11.379081 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:11.379142 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.379144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.380166 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.380167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.381010 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:11.381013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.381519 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.381520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:11.381617 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.382185 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:11.382201 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:11.383787 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:11.385678 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.385679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.385680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.385910 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.386128 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:11.386129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:11.386129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:11.386193 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:11.386466 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:11.386467 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:11.386515 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:47:11.386586 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:47:11.386684 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:47:11.386806 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.378615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.31.63.124:37795)
0809 08:47:11.378710 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.378712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.378721 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:11.378745 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.378746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.378758 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.378833 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.379244 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.380066 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.381135 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.381444 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.381515 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.383885 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.383887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.383887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.383896 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.385598 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.385993 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.385994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.385994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.386061 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.386265 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.386266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.386266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.386403 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.386471 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.386626 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.386710 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 08:47:11.388473 456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021: Bootstrap starting.
I20260809 08:47:11.389243 456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:11.389612 456 log.cc:824] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:11.390447 456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021: No bootstrap required, opened a new log
I20260809 08:47:11.391834 456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } }
I20260809 08:47:11.391971 456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 645ae096ae834f838750235f4b8ea021, State: Initialized, Role: FOLLOWER
I20260809 08:47:11.392050 456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } }
I20260809 08:47:11.392385 456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:11.392414 417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 08:47:11.392498 417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:11.393298 417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } }
I20260809 08:47:11.393551 417 leader_election.cc:288] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [CANDIDATE]: Term 1 election: Requested vote from peers 283899fb6d894182b05b622f15f07d55 (127.31.63.125:33351), 21cbce6ab5274ba8aa1d49b98cc9002c (127.31.63.124:37795)
I20260809 08:47:11.394208 642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "645ae096ae834f838750235f4b8ea021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c"
I20260809 08:47:11.394361 642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:11.394312 533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "645ae096ae834f838750235f4b8ea021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "283899fb6d894182b05b622f15f07d55"
I20260809 08:47:11.394452 533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:11.395262 642 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 645ae096ae834f838750235f4b8ea021 in term 1.
I20260809 08:47:11.395500 381 leader_election.cc:302] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21cbce6ab5274ba8aa1d49b98cc9002c, 645ae096ae834f838750235f4b8ea021; no voters:
W20260809 08:47:11.395874 747 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:11.395927 747 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:11.396198 533 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 645ae096ae834f838750235f4b8ea021 in term 1.
I20260809 08:47:11.396415 727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } } }
I20260809 08:47:11.396571 727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:11.396657 727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:11.396740 727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [term 1 LEADER]: Becoming Leader. State: Replica: 645ae096ae834f838750235f4b8ea021, State: Running, Role: LEADER
I20260809 08:47:11.396857 727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } }
I20260809 08:47:11.397311 727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 645ae096ae834f838750235f4b8ea021. Latest consensus state: current_term: 1 leader_uuid: "645ae096ae834f838750235f4b8ea021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } } }
I20260809 08:47:11.397380 727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:11.398409 353 master_runner.cc:263] No leader master found from master 645ae096ae834f838750235f4b8ea021
I20260809 08:47:11.398461 353 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:47:11.409931 758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:11.410194 758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:11.415603 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.401827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.31.63.125:33351)
0809 08:47:11.401870 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.401873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.402701 (+ 828us) server_negotiation.cc:407] Connection header received
0809 08:47:11.402803 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.402804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.402816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.402846 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.404703 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.405459 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.408305 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.408611 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.408653 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.411730 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.411731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.411732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.411740 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.414049 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.414541 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.414543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.414543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.414646 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.414857 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.414858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.414858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.415050 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.415140 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.415400 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.415484 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:47:11.417663 642 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [term 1 FOLLOWER]: Refusing update from remote peer 645ae096ae834f838750235f4b8ea021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:11.418036 753 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:11.419435 687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 645ae096ae834f838750235f4b8ea021. Latest consensus state: current_term: 1 leader_uuid: "645ae096ae834f838750235f4b8ea021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } } }
I20260809 08:47:11.419538 687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:11.419945 533 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [term 1 FOLLOWER]: Refusing update from remote peer 645ae096ae834f838750235f4b8ea021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:11.420215 753 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:11.420948 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.402167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42042 (local address 127.31.63.124:37795)
0809 08:47:11.402212 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.402214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.403735 (+ 1521us) server_negotiation.cc:407] Connection header received
0809 08:47:11.403841 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.403842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.403853 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.403888 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.406561 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.407277 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.409010 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.409326 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.409369 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.417614 (+ 8245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.417615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.417616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.417625 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.419607 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.420080 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.420081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.420081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.420137 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.420376 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.420377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.420378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.420507 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.420576 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.420775 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.420853 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:47:11.421438 709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 645ae096ae834f838750235f4b8ea021. Latest consensus state: current_term: 1 leader_uuid: "645ae096ae834f838750235f4b8ea021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } } }
I20260809 08:47:11.421550 709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:11.425843 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:11.402442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49440 (local address 127.31.63.126:35429)
0809 08:47:11.402480 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.402482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.410922 (+ 8440us) server_negotiation.cc:407] Connection header received
0809 08:47:11.411016 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.411017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.411029 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.411059 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.412031 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.412767 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.418019 (+ 5252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:11.418344 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:11.418385 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:11.422868 (+ 4483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:11.422870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:11.422870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:11.422877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:11.424491 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.424875 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.424876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.424877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.424931 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:11.425136 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:11.425136 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:11.425137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:11.425348 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:11.425447 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:11.425669 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 08:47:11.425751 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:47:11.426610 727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "645ae096ae834f838750235f4b8ea021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } } }
I20260809 08:47:11.426704 727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:11.427480 709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "645ae096ae834f838750235f4b8ea021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } } }
I20260809 08:47:11.427578 709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:11.427616 753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "645ae096ae834f838750235f4b8ea021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } } }
I20260809 08:47:11.427687 766 mvcc.cc:205] Tried to move back new op lower bound from 7316542387882352640 to 7316542387802382336. Current Snapshot: MvccSnapshot[applied={T|T < 7316542387882352640}]
I20260809 08:47:11.427721 753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:11.428143 763 mvcc.cc:205] Tried to move back new op lower bound from 7316542387882352640 to 7316542387802382336. Current Snapshot: MvccSnapshot[applied={T|T < 7316542387882352640}]
I20260809 08:47:11.428246 687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "645ae096ae834f838750235f4b8ea021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645ae096ae834f838750235f4b8ea021" member_type: VOTER last_known_addr { host: "127.31.63.126" port: 35429 } } peers { permanent_uuid: "283899fb6d894182b05b622f15f07d55" member_type: VOTER last_known_addr { host: "127.31.63.125" port: 33351 } } peers { permanent_uuid: "21cbce6ab5274ba8aa1d49b98cc9002c" member_type: VOTER last_known_addr { host: "127.31.63.124" port: 37795 } } }
I20260809 08:47:11.428336 687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:11.428510 764 mvcc.cc:205] Tried to move back new op lower bound from 7316542387882352640 to 7316542387802382336. Current Snapshot: MvccSnapshot[applied={T|T < 7316542387882352640}]
I20260809 08:47:11.430161 758 catalog_manager.cc:1347] Generated new cluster ID: 6c62c65f72404ffc8adcab8c228dac65
I20260809 08:47:11.430213 758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:11.449460 758 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:11.450311 758 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:11.455173 758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 645ae096ae834f838750235f4b8ea021: Generated new TSK 0
I20260809 08:47:11.455531 758 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:47:11.483580 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.482154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59712 (local address 127.31.63.125:33351)
0809 08:47:11.482203 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.482205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.482851 (+ 646us) server_negotiation.cc:407] Connection header received
0809 08:47:11.482959 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.482960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.482971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.483001 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.483501 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59712: BlockingRecv error: recv got EOF from 127.0.0.1:59712 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 08:47:11.484793 683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.482534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42050 (local address 127.31.63.124:37795)
0809 08:47:11.482575 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.482577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.484235 (+ 1658us) server_negotiation.cc:407] Connection header received
0809 08:47:11.484336 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.484338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.484349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.484380 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.484713 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42050: BlockingRecv error: recv got EOF from 127.0.0.1:42050 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:47:11.485571 464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.483905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49454 (local address 127.31.63.126:35429)
0809 08:47:11.483951 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.483953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.484997 (+ 1044us) server_negotiation.cc:407] Connection header received
0809 08:47:11.485098 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.485099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.485111 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.485140 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.485482 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49454: BlockingRecv error: recv got EOF from 127.0.0.1:49454 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 08:47:11.487671 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.486907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59718 (local address 127.31.63.125:33351)
0809 08:47:11.486943 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.486945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.487161 (+ 216us) server_negotiation.cc:407] Connection header received
0809 08:47:11.487264 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.487265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.487285 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.487318 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.487617 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59718: BlockingRecv error: recv got EOF from 127.0.0.1:59718 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:47:11.490828 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.489944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.31.63.125:33351)
0809 08:47:11.489978 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.489980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.490274 (+ 294us) server_negotiation.cc:407] Connection header received
0809 08:47:11.490393 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.490394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.490405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.490438 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.490780 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59734: BlockingRecv error: recv got EOF from 127.0.0.1:59734 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:47:11.494916 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.494010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.31.63.125:33351)
0809 08:47:11.494052 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.494054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.494294 (+ 240us) server_negotiation.cc:407] Connection header received
0809 08:47:11.494393 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.494394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.494407 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.494435 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.494853 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59746: BlockingRecv error: recv got EOF from 127.0.0.1:59746 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:47:11.500944 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.500114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.31.63.125:33351)
0809 08:47:11.500153 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.500155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.500409 (+ 254us) server_negotiation.cc:407] Connection header received
0809 08:47:11.500509 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.500510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.500523 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.500556 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.500892 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59754: BlockingRecv error: recv got EOF from 127.0.0.1:59754 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:47:11.511037 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.510161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.31.63.125:33351)
0809 08:47:11.510204 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.510206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.510473 (+ 267us) server_negotiation.cc:407] Connection header received
0809 08:47:11.510569 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.510571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.510582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.510611 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.510980 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59762: BlockingRecv error: recv got EOF from 127.0.0.1:59762 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:47:11.529098 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.528297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.31.63.125:33351)
0809 08:47:11.528347 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.528349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.528510 (+ 161us) server_negotiation.cc:407] Connection header received
0809 08:47:11.528581 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.528582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.528595 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.528660 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.529024 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59778: BlockingRecv error: recv got EOF from 127.0.0.1:59778 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 08:47:11.563231 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.562394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.31.63.125:33351)
0809 08:47:11.562448 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.562451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.562603 (+ 152us) server_negotiation.cc:407] Connection header received
0809 08:47:11.562683 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.562684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.562698 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.562778 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.563151 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59792: BlockingRecv error: recv got EOF from 127.0.0.1:59792 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 08:47:11.629240 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.628396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.31.63.125:33351)
0809 08:47:11.628468 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.628470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.628600 (+ 130us) server_negotiation.cc:407] Connection header received
0809 08:47:11.628702 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.628704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.628724 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.628803 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.629167 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59808: BlockingRecv error: recv got EOF from 127.0.0.1:59808 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 08:47:11.759552 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:11.758815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.31.63.125:33351)
0809 08:47:11.758874 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:11.758876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:11.758948 (+ 72us) server_negotiation.cc:407] Connection header received
0809 08:47:11.759008 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:11.759009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:11.759020 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:11.759062 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:11.759473 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59818: BlockingRecv error: recv got EOF from 127.0.0.1:59818 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:47:12.017519 576 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:12.016952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59830 (local address 127.31.63.125:33351)
0809 08:47:12.017034 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:12.017036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:12.017046 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:12.017072 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:12.017073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:12.017084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:12.017135 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:12.017467 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59830: BlockingRecv error: recv got EOF from 127.0.0.1:59830 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 08:47:12.333292 703 catalog_manager.cc:1259] Loaded cluster ID: 6c62c65f72404ffc8adcab8c228dac65
I20260809 08:47:12.333374 703 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c: loading cluster ID for follower catalog manager: success
I20260809 08:47:12.334263 703 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c: acquiring CA information for follower catalog manager: success
I20260809 08:47:12.334602 703 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 21cbce6ab5274ba8aa1d49b98cc9002c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:47:12.371886 723 catalog_manager.cc:1259] Loaded cluster ID: 6c62c65f72404ffc8adcab8c228dac65
I20260809 08:47:12.371971 723 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55: loading cluster ID for follower catalog manager: success
I20260809 08:47:12.372839 723 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55: acquiring CA information for follower catalog manager: success
I20260809 08:47:12.373157 723 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 283899fb6d894182b05b622f15f07d55: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:47:12.533386 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:12.530812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.31.63.125:33351)
0809 08:47:12.530968 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:12.530971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:12.531038 (+ 67us) server_negotiation.cc:407] Connection header received
0809 08:47:12.531104 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:12.531105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:12.531161 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:12.531232 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:12.531395 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:12.531899 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:12.532639 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:12.532834 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:12.532883 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:12.533036 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:12.533146 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:12.533204 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:12.533241 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 08:47:12.533260 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260809 08:47:12.537838 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:12.535266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42054 (local address 127.31.63.124:37795)
0809 08:47:12.535388 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:12.535390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:12.535400 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:12.535492 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:12.535493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:12.535542 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:12.535628 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:12.535763 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:12.536241 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:12.536973 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:12.537188 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:12.537264 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:12.537422 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:12.537587 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:12.537661 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:12.537698 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 08:47:12.537720 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":50,"threads_started":1}
I20260809 08:47:12.542768 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:12.539419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.31.63.125:33351)
0809 08:47:12.539458 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:12.539461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:12.539691 (+ 230us) server_negotiation.cc:407] Connection header received
0809 08:47:12.539964 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:12.539965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:12.539975 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:12.540029 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:12.540188 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:12.540919 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:12.541976 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:12.542238 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:12.542308 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:12.542508 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:12.542546 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:12.542602 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:12.542660 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 08:47:12.542686 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:47:12.542829 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:12.539452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49468 (local address 127.31.63.126:35429)
0809 08:47:12.539621 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:12.539624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:12.539736 (+ 112us) server_negotiation.cc:407] Connection header received
0809 08:47:12.539826 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:12.539827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:12.539883 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:12.539974 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:12.540155 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:12.540655 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:12.541764 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:12.542083 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:12.542159 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:12.542373 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:12.542499 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:12.542586 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:12.542679 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 08:47:12.542704 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260809 08:47:12.543059 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:12.539659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.31.63.124:37795)
0809 08:47:12.539701 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:12.539704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:12.539814 (+ 110us) server_negotiation.cc:407] Connection header received
0809 08:47:12.539874 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:12.539875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:12.539887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:12.539933 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:12.540100 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:12.540821 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:12.541830 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:12.542256 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:12.542311 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:12.542520 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:12.542551 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:12.542605 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:12.542963 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 08:47:12.542988 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:47:12.544909 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 353
I20260809 08:47:12.556842 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 470
I20260809 08:47:12.567974 31997 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskslFlM2/build/release/bin/kudu with pid 580
2026-08-09T08:47:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3141 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6178 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (313802 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS